Peter Marklund

Peter Marklund's Home

Peter on Rails

Lessons Learned in Ruby on Rails and related technologies. Subscribe

Tue November 20, 2012
Programming

We+

This autumn I have joined We+ - a new exciting startup in the exercise and health area. Our product uses the power of small groups and positive peer pressure to promote exercise among employees. We have already built a first version of the system and done Alpha testing and this week we are kicking off a pilot with two large swedish companies. Exciting!

Make a Comment

Sat June 30, 2012
Programming

Creating and Deploying an EdgeRails (Rails 4) Application to Heroku

I wanted to create a new Rails app based on EdgeRails (Rails 4) and I didn't find much when I googled around so I ended up creating a Gist on Github and answering the corresponding question on Stack Overflow.

I'm really happy to see how Rails continues to evolve through thousands of small improvements, many of which are features which I know would have been useful in projects I have worked on in the past. I hope to be able to cover some of the new stuff in Rails 4 later.

1 Comment

Tue September 20, 2011
Programming

Explore the Future of Tablet Publishing With Mag+

We have a great job opening right now for Ruby developers at Mag+. Mag+ helps bring magazines and other publications (such as the IKEA catalogue) to the iPad and other tablets. Please check out the job ad if you are interested.

1 Comment

Fri September 02, 2011
Programming

Course Material for a Two Day Introductory Ruby Course

I'm giving a series of two day introductory Ruby courses to C++ programers at Ericsson here in Stockholm and I've made the course material available on Github. The course material includes slides (keynote and pdf) as well as Ruby code examples. For the exercise part I am relying on the Ruby Koans and they have been much appreciated.

I enjoy being back in the teacher role and I hope to be able to teach courses like this many times again in the future.

Make a Comment

Thu November 25, 2010
Programming

Rails Tip: Inserting NULL to the Database Instead of Empty Strings

The value NULL in a relational database represents the absence of a value. Empty text fields and text areas in HTML forms on the other hand get submitted in Rails as empty strings. This means you can easily end up with empty strings in the database where you would expect NULL values. I came up with the following workaround for our ActiveRecord models:

5 Comments

Thu November 18, 2010
Programming

Rails 3.0.3 Backwards Incompatible for File Uploads

We upgraded from Rails 3.0.1 to Rails 3.0.3 yesterday and sadly this broke file uploads for us in production. What do we conclude from this other than that we are missing integration tests for file uploads? Well, one noteworthy thing is that the Rails 3.0.3 release is not quite as backwards compatible as announced, at least not when it comes to file uploads. In Rails file fields in multipart forms are exposed in the params hash as an ActionDispatch::Http::UploadedFile object. As of a commit by tenderlove on the 5:th of October this class no longer inherits from Tempfile but instead delegates to Tempfile (i.e. tenderlove favored composition over inheritance like the GoF prescribes). Unfortunately, the UploadedFile class only delegates a handful of methods and not two important methods that we happened to be using, namely open and path. So now, for example when getting the path of the tempfile we have to fetch the tempfile first:

  params[:my_file].tempfile.path

9 Comments

Mon November 15, 2010
Programming

Testing Rails Migrations

Is it worthwhile testing Rails ActiveRecord migrations? After all, they are only intended to be run once so regression testing isn't an issue. I honestly haven't tested my migrations much in the past but I recently decided to give it a try. I was surprised by the fact that it wasn't very different from testing any other part of my application. My test didn't end up having very good coverage so I still needed to test the migration manually. As usual when writing tests, I found that it drove a series of extract method refactorings. I went from having all code in the up method to having five shorter methods. A different approach to migration testing is to add sanity checks at the end of migrations that output an error message in production if the outcome of the migration wasn't what was expected.

Make a Comment

Fri November 05, 2010
Programming

Heroku Deploys With Rollbacks and Changelog

One of the features I miss from Capistrano is the ability to easily do rollbacks when deploying to Heroku. What you can easily do though is git tag your releases and then do a rollback by pushing the previous release tag. I've created a RubyGem called heroku_release that does this. The gem has a few additional features such as the ability to generate a CHANGELOG file from the release tags and their comments. I also use it to generate a version file so that I can check on the live server what version of the code it is running.

It's interesting to note that Heroku is apparently working on supporting release management and logging - two problems I have ended up rolling my own solutions for recently. I look forward to seeing what they have come up with.

Make a Comment

Fri November 05, 2010
Programming

Ruby Debug Printouts

It can get really tiresome and repetitive to do debug printouts with puts. I've created a simple RubyGem called debug_log that gives you a convenient way of evaluating and printing variables and other Ruby expressions that you want to debug. Here is an example:

It's funny how I created this gem pretty much at the same time as Niclas Nilsson created his dp gem. I owe the approach to patching the binding object to Niclas. I think that is a beautiful solution as it avoids you having to pass the binding object as an argument.

2 Comments

Mon November 01, 2010
Programming

Ruby Testing: Avoid Stubbing Non-Existent Methods with Mocha

Mocks and stubs can be fragile and come back and bite you when they can get out of sync with your code. One way this can happen is that a method is renamed or you misspell the method name. To avoid this issue you can configure Mocha to disallow stubbing and mocking of non-existent methods. I've come up with two new methods for the cases where you are working with messages implemented with method_missing:

It is interesting to note that this extension of Mocha is possible because of its flexible and clever design. Mocha was recently updated to version 0.9.9. I am grateful to James Mead (Floehopper) for providing this excellent testing library.

2 Comments

Fri October 29, 2010
Programming

Request Log - RubyGem for Logging Rack (Rails) Web Requests to MongoDB

Prompted by the fact that Heroku doesn't keep the Rails request logs around I went out looking for a logging solution. What I've ended up with is Request Log - a simple RubyGem for logging web requests to MongoDB.

My experiences with logging to MongoDB so far have been very positive. I see big potential in logging web requests to a database. The reason MongoDB is so well suited for the task is its high performance and strong query capabilities. This allows you to do advanced queries such as "give me all requests in this time period, with this response, status, this execution time, these parameters etc.". Each web request becomes a document in MongoDB and if you choose your database fields wisely you have a great tool at your disposal for statistics, monitoring, and debugging etc.

I'm curious to see how we'll be able to design and use our web request logs in the project I'm currently in. I'll report back here any interesting findings that we make.

1 Comment

Mon October 25, 2010
Programming

Rails Presentation: Minimizing Library Dependencies

I gave a presentation tonight entitled "Minimizing Library Dependencies" at the Stockholm Ruby User Group (SHRUG) meeting. The event was hosted by MediaPilot and sponsored (with beer) by Auktionskompaniet and it turned out to be a huge success with 73 registered attendees, great presentations and atmosphere. I talked to David Wennergren about hosting the next meetup and our ambition is to have about one per quarter. It's great to see the community coming to life again!

The slides for my presentation are hosted on Github and are also available here.

Make a Comment

Sat May 29, 2010
Programming

Migrating RSpec to Mocha

Over at the MyNewsdesk developer blog:

1 Comment

Tue May 11, 2010
Programming

PostgreSQL Unreliable Default Sort Order and Random Rails Test Failures

More good stuff from the MyNewsdesk Developer Blog:

Make a Comment

Wed May 05, 2010
Programming

Rails 2.3.2 Bug and Rails Cache Enable/Disable

Two new posts from the MyNewsdesk Developer Blog:

Make a Comment

Tue April 20, 2010
Programming

New Plugins For Rails Model Caching: Cachable Model

As part of an ongoing effort to offload out database at MyNewsdesk I have released a new Rails plugin called Cachable Model. The plugin is similar to an older plugin called Cached Model in that it basically caches primary key id lookups for ActiveRecord models. The Cachable Model plugin has an extra feature that lets you cache lookups by other unique columns as well. Here is an example:

3 Comments

Mon January 18, 2010
Programming

Newsdesk Developer Blog - Back in Business

Now that I'm back working at Newsdesk I've started posting in our developer blog again:

Make a Comment

Wed January 06, 2010
Programming

Book Tip: The Mythical Man Month

Fred Brooks is a Computer Science professor who managed 5000 man-year IT projects at IBM in the sixties. His words carry a lot of weight and he certainly has something profound to say about software engineering. The central theme of the book is that of conceptual integrity. It's about the need to have a single architect, a master mind, who oversees all development and makes sure all the parts fit together. "A clean, elegant programming product must present to each of its users a coherent mental model of the application". The ideal scenario for conceptual integrity naturally is having a single programmer. The problem is that some systems are so big that in order to finish them before they are obsolete you need a large number of developers. Much of the book is spent discussing this difficult problem. How do you organize huge developer teams?

Read More

Mon September 07, 2009
Programming

Teaching a Three Day Ruby on Rails Course in Rome

This weekend I taught a three day Ruby on Rails course here in Rome in Italy. It was a great experience and the people down here have shown the greatest hospitality and have taken very well care of me.

The format of the course was like a workshop with a small group of participants in a private and relaxed setting. I used my course material as a starting point and a road map but then improvised a lot and ended up doing a lot of hands on programming. Basically the course was divided into three parts:

  1. Ruby and Rails fundamentals. A lot of time was spent on the programming language itself and we toured the features of Ruby by live coding in TextMate.
  2. Writing a demo application. Similar to the AWR book, I built the basics for a store application, including file upload, advanced ActiveRecord associations, and deployment to a VPS with Capistrano.
  3. The participants got to work on different features in the application. Time was also spent code reviewing the participants own Rails applications.

Overall I must say the course was quite a success. Every time I teach a course I look for ways to make my courses more interactive, more hands on, and more tailored to the needs of the participants.

I also had a great time outside the course in Rome - a city that I fell in love with immediately.

3 Comments

Mon August 17, 2009
Programming

Rails Counter Cache Updates Leading to MySQL Deadlock

I've gotten a few error messages lately where a plain vanilla ActiveRecord counter cache update (update_counters_without_lock method) has lead to an error being thrown from Mysql - "Mysql::Error: Deadlock found when trying to get lock; try restarting transaction: UPDATE `events` SET `attendees_count` = COALESCE(`attendees_count`, 0) + 1 WHERE (`id` = 1067)".

It seems someone else has tried to report this as a bug but Mysql is saying that it's a feature and is referring to the lock modes documentation. There is some interesting info on deadlocks in InnoDB over at rubyisms. I haven't had time to dig into the theory though. Has anybody else had this issue? What can be done about it (other than switch to PostgreSQL)?

4 Comments

Fri June 12, 2009
Programming

Rails Testing: To Stub or Not to Stub

Over the last couple of years testing has been a controversial topic in Rails. In the beginning though Rails was of course opinionated and gave us model and controller tests along with fixtures for testing our apps. Then integration tests were added by Jamis. Some started using browser testing with tools such as Selenium and Watir. Then the whole RSpec movement swept in with a new terminology, a heavier use of mocking and stubbing, and more isolated testing of the different layers of the MVC stack. One of the most important trends right now seems to be to do "Outside In TDD" with Cucumber and webrat. There are alternatives to RSpec like Shoulda and a move towards simpler tools such as Jeremmy Mcannalys Context and Matchy libraries. There are a number of Factory libraries for replacing fixtures. Maybe the most important controversy over the years has been on whether the database should be stubbed out or not. One of the most common arguments for stubbing out the database is to keep the test execution time low.

Personally I've always been a stubbing sceptic. Given all the changes in the testing landscape it's interesting to read that at Thoughtworks there is a movement away from stubbing and back to where it all started:

"As the teams became familiar with using method stubbing, they used it more and more - falling into the inevitable over-usage where unit tests would stub out every method other than the one being tested. The problem here, as often with using doubles, is brittle tests. As you change the behavior of the application, you also have to change lots of doubles that are mimicking the old behavior. This over-usage has led both teams to move away from stubbed unit tests and to use more rails-style functional tests with direct database access."

2 Comments

Fri June 12, 2009
Programming

Rails Development Database Setup Without Migrations

I posted over at the Newsdesk developer blog about Rails Development Database Setup Without Migrations.

Make a Comment

Tue April 28, 2009
Programming

The Newsdesk Developer Blog

I have been doing a bit of blogging over at the Newsdesk developer blog. My two most recent posts are on shelling out to external programs and eager loading of application classes to save STI.

Make a Comment

Fri April 24, 2009
Programming

Rails Gotcha: ActiveRecord##Base.valid?, errors.empty?, and before_validation Callbacks

The ActiveRecord::Callbacks module (that depends on ActiveSupport::Callbacks) defines a multitude of before and after callbacks for the lifecycle of your ActiveRecord objects. Similar to how before filters in controllers work, if a before_validation callback method returns false, the save process will be aborted. Here is what the API documentation says:

# If the returning value of a +before_validation+ callback can be evaluated to +false+,
# the process will be aborted and Base#save< will return +false+.
# If Base#save! is called it will raise a ActiveRecord::RecordInvalid exception.
# Nothing will be appended to the errors object.

# If a before_* callback returns +false+, all the later callbacks and the associated
# action are cancelled. If an after_* callback returns
# +false+, all the later callbacks are cancelled. Callbacks are generally run in the 
# order they are defined, with the exception of callbacks
# defined as methods on the model, which are called last.

What this means is that if a before_validation callback returns false, then save and valid? will both return false but errors.empty? will return true. Yes, you read correctly, there are no validation errors but the record is not valid. Also, note that the valid? method defined in the ActiveRecord::Base class will never even be invoked.

3 Comments

Fri April 17, 2009
Programming

Rails Tip: Running Tests with Verbose Output

If you want to run your Rails tests with verbose output you can use the TESTOPTS argument like this:

rake test TESTOPTS="-v"

Read More | 4 Comments

Fri April 03, 2009
Programming

Plugin Stack Traces Missing in Rails 2.3.2

I endured a nightmarish debugging session yesterday due to Rails no longer showing proper stack traces from exceptions in plugin code. I've verified this across different Rails apps and it seems to actually be the case that with Rails 2.3.2 you no longer get a stack trace from plugins. Has anybody else had issues with this? Any pointers to where in the Rails code exception handling and stack traces are dealt with and how it could be fixed? I posted about this issue on the Rails core list but haven't received a reply yet.

3 Comments

Thu April 02, 2009
Programming

You're a SlideShare RockStar

SlideShare sent me an email saying I'm a rockstar because my Ruby on Rails 101 slides have been getting so popular. I'm really glad that I decided to share my slides and SlideShare has been a great boost in reaching more people. All the great feedback that I get from people makes all the work that went into the slides more than worthwhile. I continue to be fascinated with how rewarding it can be to share information on the web.

2 Comments

Tue March 31, 2009
Programming

Rails Tip: Using Rakismet to Stop Spam

I am trying out Rakismet now for this blog to prevent comment spam - an issue that has been bugging me for a long time and that has been getting worse lately. Setting up the Rakismet plugin was a breeze. Very nice! Now, let's hope it actually stops the spammers. We use Rakismet at Newsdesk and I think it's worked out really well there.

7 Comments

Tue March 31, 2009
Programming

Really Simple Rails Log Rotatation

I always used logrotate Linux tool to setup log rotation for my Rails apps which has worked fine although it required finding some external config file and understanding its config options and syntax. I never new log rotation could be set up by adding this one line to config/environments/production.rb right in your app:

config.logger = Logger.new(config.log_path, 50, 1.megabyte)

Sweet!

3 Comments

Sun March 29, 2009
Programming

Upgrading from Rails 1.2.3 to Rails 2.3.2

Have you ever tried upgrading Rails from version 1 to version 2? If not, you're in for a treat. Seriously though, it's not that bad, but depending on the size of your app, the number of plugins you use, and how badly you've patched Rails, your mileage may vary.

Here are some notes from an upgrade of this little weblog app just now:

Read More | 13 Comments

Tue March 24, 2009
Programming

Contributing to Rails Core

At Newsdesk we recently upgraded to Rails 2.3 and ran into a Rails bug triggered by multiple post requests in tests. I was curious to figure out why the bug occured and since I was already familiar with the Rails testing code (in test_process.rb) I was able to nail down the problem pretty quickly. Usually I would probably stopped there, but my colleague Richard encouraged me to submit a Rails patch so I walked the extra mile and wrote a little unit test for my fix. I then followed the contribution instructions to submit an issue in Lighthouse with a patch file followed by the recommended post to the Rails core mailing list. The issue was quickly assigned to Joshua Peek in the Rails core team and a few days later it was applied. It is really encouraging to see how easy it can be to contribute to Rails and that the core team actually picks up a lot of patches and applies them if only they are submitted properly. This really illustrates the power of of open source in general and Git and Rails in particular.

1 Comment

Mon March 23, 2009
Programming

Rails Tip: JavaScript Validation and Testing

JavaScript test coverage is pretty non-existant in most Rails projects and if your application has a lot of JavaScript code this can become a real problem. But it doesn't have to be that way. One thing you can do just to get some basic syntax and style checking of your JavaScript code is to run it through JavaScript Lint. I've added a simple rake task to run all javascript files through the validator:

namespace :test do
  namespace :javascripts do
    desc "Validate all javascript files with javascript lint - assumes jsl on the command line"
    # Visit http://www.javascriptlint.com to download and install the jsl command line tool
    task :validate do
      total_errors = 0
      Dir[File.join(File.dirname(__FILE__), "..", "..", "public", "javascripts", "*.js")].each do |file_path|
        print "#{file_path} ... "
        result = `jsl -process #{file_path}`
        n_errors, n_warnings = result.match(/(\d+) error\(s\), (\d+) warning\(s\)$/).to_a[1, 2].map(&:to_i)
        
        if n_errors > 0
          puts "FAILED"
          puts result
        else
          puts "OK (#{n_warnings} warnings)"
        end
        total_errors += n_errors
      end

      print "TEST RESULT: "
      if total_errors > 0
        puts "FAILURE - #{total_errors} errors"
      else
        puts "OK"
      end
    end
  end
end

In addition to syntax checking I recommend trying out Dr Nic's javascript_test plugin. To get it to work with the latest prototype I had to download JsUnitTest and use that instead of unittest.js.

3 Comments

Tue March 10, 2009
Programming

Rails Tip: Migrate Your Database to UTC

UPDATE: Adam Meehan pointed out that my migration didn't work with DST, i.e. different UTC offsets for datetimes at different points of the year. I updated my migration to use a UTC conversion in the database (leading to a variable interval) instead of using a fixed interval adjustment.

If you want to make use of the timezone support in Rails 2.1 and later you'll need to migrate any existing times that you have in your db to UTC. Here is a migration for PostgreSQL I wrote to do that (you'll probably need to adjust it to work on MySQL):

# This migration will work with DST. Because of DST, if you have your datetimes
# spread across the year they will have different offset, i.e. in Stockholm we are UTC+1 usually
# but UTC+2 in the summer.
class MigrateDatabaseToUtc < ActiveRecord::Migration
  COLUMN_TYPES = [:datetime, :timestamp]

  def self.up
    adjust(:up)
  end

  def self.down
    adjust(:down)
  end
  
  def self.adjust(direction)
    connection = ActiveRecord::Base.connection
    connection.tables.each do |table|
      timestamp_columns = connection.columns(table).select do |column|
        COLUMN_TYPES.include?(column.type)
      end

      sign = (direction == :down ? "+" : "-")
      update_clause = timestamp_columns.map do |column|
        "#{column.name} = (#{column.name} #{sign} ((#{column.name} AT TIME ZONE 'UTC')-#{column.name}))"
      end.join(", ")

      execute "UPDATE #{table} SET #{update_clause}" unless update_clause.blank?
    end    
  end
end

When I originally wrote this migration I used a fixed interval adjustment (+1 for Stockholm), i.e. the same approach that Simon Harris uses. However, as Adam Meehan points out in the comments this doesn't work with DST so I adjusted to using a AT TIME ZONE 'UTC' conversion instead that will result in a DST dependent interval. Thanks Adam for pointing this out!

7 Comments

Fri March 06, 2009
Programming

Ruby on Rails 101 Slides Updated for Course in Italy

Last week I had the pleasure of teaching a five day introductory course at Tiscali in Cagliari/Itali (Sardinia). The course outline was similar to the course I held back in 2007 but in this case the lectures were half day with the afternoons dedicated to exercises and questions. This format worked out pretty well and I think it's a good idea to have half of the time devoted to hands-on training.

I've updated my old slides for Rails 2.3 and rewritten them in HTML (using S5/Codex). You can view the slides here. The source code for the slides is available on GitHub. Feel free to send feedback and reuse the slides and make any corrections and improvements that you see fit. Thanks!

1 Comment

Wed January 21, 2009
Programming

Translate: New I18n Rails Plugin with Nice Web UI

Today Newsdesk released the Translate plugin that provides a nice web UI for doing translations. The plugin mounts a web UI at /translate where you can list and translate I18n texts. Translations are written directly to YAML files stored at the default location under config/locales. Check out the post at the Newsdesk blog and the README file on Github for more details.

1 Comment

Tue January 06, 2009
Programming

Rails Timezone Gotcha: ActiveRecord::Base.find does not convert Time objects to UTC

With the timezone support introduced in Rails 2.1 the idea is that all dates in the database are stored in UTC and all dates in Ruby are in a local timezone. The local timezone can be specified by config.timezone in environment.rb or set to the user timezone with Time.zone= in a before filter. Typicaly, when reading/writing from/to the database ActiveRecord will transparently convert time attributes back and forth to UTC for you. However, there is a gotcha with datetimes in ActiveRecord::Base.find conditions. They will only be converted to UTC for you if they are ActiveSupport::TimeWithZone objects, not if they are Time objects. This means that you are fine if you use Time.zone.now, 1.days.ago, or Time.parse("2008-12-23").utc, but not if you use Time.now or Time.parse("2008-12-23"). Example:

  Event.all(:conditions => ["start_time > ?", Time.zone.now])
  Event.all(:conditions => ["start_time > ?", Time.parse("2008-12-23").utc])

Apparently this issue has been reported and marked as invalid. I think it's quite unfortunate that ActiveRecord doesn't do this conversion for us. I suspect other application developers will be bitten by this as well. The difference in behaviour between Time and TimeWithZone objects boils down to the to_s(:db) call:

>> Time.now.to_s(:db)
Time.now.to_s(:db)
=> "2009-01-06 17:52:19"
>> Time.zone.now.to_s(:db)
Time.zone.now.to_s(:db)
=> "2009-01-06 16:52:23"

One way to fix the issue would be to monkey patch the Quoting module in ActiveRecord like this:

module ActiveRecord
  module ConnectionAdapters # :nodoc:
    module Quoting
      # Convert dates and times to UTC so that the following two will be equivalent:
      # Event.all(:conditions => ["start_time > ?", Time.zone.now])
      # Event.all(:conditions => ["start_time > ?", Time.now])
      def quoted_date(value)
        value.respond_to?(:utc) ? value.utc.to_s(:db) : value.to_s(:db)
      end
    end
  end
end

However I'm not sure that this is a good idea and that it won't break anything else. I've at least verified that it doesn't break assignment of ActiveRecord attributes.

18 Comments

Thu December 04, 2008
Programming

MySQL Performance

Every now and then I like to pick on MySQL and it's become something of a running theme in this blog. My session table for this blog (which runs on Ruby on Rails of course) had grown too big so I needed to clean it up. I just didn't expect it to take this long:

mysql> select count(*) from sessions;
+----------+
| count(*) |
+----------+
|   545797 | 
+----------+
1 row in set (2.89 sec)

mysql> delete from sessions;
Query OK, 545801 rows affected (5 min 46.67 sec)

Should it really take 3 seconds to count half a million rows? I wonder if PostgreSQL would deliver better performance in this instance. As soon as I find the time I will switch over my Rails/MySQL based web application Simple Signup to PostgreSQL. I have a strong personal preference for PostgreSQL over MySQL. I'm curious to see what the transition will be like though.

7 Comments

Mon October 13, 2008
Programming

Rails Tip: SEO Friendly URLs (Permalinks)

There are several plugins already out there that can turn the typical Rails show path of /articles/show/1 into something more search engine friendly. I decided to roll my own implementation and it turned out to be fairly easy. My solution relies on the Slugalizer library by Henrik Nyh. First, I make sure we can turn any string into something URL friendly by patching the String class:

class String
  # Convert String to something URL and filename friendly
  def to_uri(max_size = 150, separator = "-")
    no_slashes = self.gsub(%r{[/]+}, separator)
    Slugalizer.slugalize(no_slashes.swedish_sanitation, separator).truncate_to_last_word(max_size, separator)
  end

  # We need this for SEO/Google reasons sincen å should become aa and Slugalizer translates å to a.
  def swedish_sanitation
    dup = self.dup
    dup.gsub!('å', 'aa')
    dup.gsub!('Å', 'aa')
    dup.gsub!('ä', 'ae')
    dup.gsub!('Ä', 'ae')
    dup.gsub!('ö', 'oe')
    dup.gsub!('Ö', 'oe')
    dup.gsub!('é', 'e')
    dup.gsub!('É', 'e')
    dup
  end
  
  def truncate_to_last_word(length, separator = "-")
    dup = self.dup
    if dup.size > length
      truncated = dup[0..(length-1)]
      if truncated.include?(separator)
        truncated[/^(.+)#{separator}/, 1]
      else
        truncated
      end
    else
      dup
    end
  end
end

All I have to do in my ActiveRecord model then is to override the to_param method:

  def permalink
    if name.present?
      "#{id}-#{name.to_uri}"
    else
      id
    end
  end

  def to_param
    permalink
  end

ActiveRecord will automatically ignore any non-digit characters after the leading digits in an id that you pass to it, but just to be on the safe side I added a before_filter to my application controller that will convert permalinks to proper ids:

  def convert_permalink_to_id
    if params[:id].present? && params[:id] =~ /\D/
      params[:id] = params[:id][/^(\d+)/, 1]
    end
  end

Credit for parts of this code goes to my cool programmer colleagues over at Newsdesk.se.

6 Comments

Mon October 06, 2008
Programming

Rails Hack: Auto Strip ActiveRecord Attributes

We have a user who unintentially enters a space after his email address. It seems that a lot of times it makes sense to automatically strip ActiveRecord model attributes before they are validated. Inspired by this post I came out with my own auto_strip method that adds a before validation callback which seems less intrusive than redefining the attribute setter method:

# Sometimes users accidentally enter space before or after text in text fields. Let's not punish them
# with an error message for this.
module ActiveRecord
  class Base
    def self.auto_strip(*attributes)
      attributes.each do |attribute|
        before_validation do |record|
          record.send("#{attribute}=", record.send("#{attribute}_before_type_cast").to_s.strip) if record.send(attribute)
        end
      end
    end
  end
end

Now in my ActiveRecord model I can say for example:

  auto_strip :price, :vat, :max_tickets

Maybe auto stripping would be useful as an option to the Rails validation macros?

1 Comment

Wed September 17, 2008
Programming

Rails I18n: Array#to_sentence and :skip_last_comma

The Array#to_sentence method in ActiveSupport joins an array into a string with a locale dependent connector such as "and" for english and "och" for swedish. The sentence connector is determined by the I18n message support.array.sentence_connector. The issue is that in english you have a comma before the last connector whereas in swedish you don't. The existance of the last comma is set with the :skip_last_comma option. Ideally we would like this to always be true for the swedish locale. Therefore I added the I18n key support.array.sentence_skip_last_comma and patched the to_sentence method:

class ::Array
  alias_method :to_sentence_orig, :to_sentence
  def to_sentence(options = {})
    extra_options = {}
    skip_last_comma = I18n.translate(:'support.array.sentence_skip_last_comma')
    extra_options[:skip_last_comma] = skip_last_comma if skip_last_comma !~ /translation missing: /
    to_sentence_orig(options.reverse_merge(extra_options))
  end
end

Here is the corresponding spec:

    it "can join array elements with Array#to_sentence" do
      with_locale("en-US") { ["knatte", "fnatte", "tjatte"].to_sentence.should == "knatte, fnatte, and tjatte" }
      with_locale("sv-SE") { ["knatte", "fnatte", "tjatte"].to_sentence.should == "knatte, fnatte och tjatte" }
      with_locale("sv-SE") do
        ["knatte", "fnatte", "tjatte"].to_sentence(:skip_last_comma => false).should == "knatte, fnatte, och tjatte"
      end
    end

1 Comment

Wed September 10, 2008
Programming

Upgrading to Rails 2.2 and Drinking the I18n Koolaid

The other day I got tired of waiting for the Rails 2.2 release and upgraded my application from Rails 2.1 to edge. The process went a little something like this:

rake rails:freeze:edge

cd vendor/plugins
rm -rf rspec
rm -rf rspec_on_rails
git clone git://github.com/dchelimsky/rspec.git rspec_on_rails
git clone git://github.com/dchelimsky/rspec-rails.git
rm -rf rspec/.git
rm -rf rspec-rails/.git
cd ../../
./script/generate rspec

At this point I ran my Test::Unit tests and RSpec specs with rake and ended up with a single failing test and a bunch of warnings:

Overall, it was fairly easy to transition from 2.1 to edge. The really interesting part though was I18n. Thanks to the new I18n support in Rails I got to throw out the Simple Localization plugin that apparently is no longer supported, as well as my own hack to get error messages to be localized. I used the I18n demo app as a starting point and added config/locales/sv-SE.yml. It turned out the structure of I18n keys had changed a little since the demo app was written. You can use my file as a starting point or probably better, copy the following files into your single locales file and translate those:

vendor/rails/actionpack/lib/action_view/locale/en-US.yml
vendor/rails/activerecord/lib/active_record/locale/en-US.yml
vendor/rails/activesupport/lib/active_support/locale/en-US.yml

I source the locales file from config/initializers/i18n.rb:

I18n.default_locale = 'en-US'

LOCALES_DIRECTORY = "#{RAILS_ROOT}/config/locales/"
locale_files = Dir["#{LOCALES_DIRECTORY}/*.{rb,yml}"]
LOCALES_AVAILABLE = (locale_files.collect do |locale_file|
  File.basename(File.basename(locale_file, ".rb"), ".yml")
end + ['en-US']).uniq
locale_files.each { |locale_file| I18n.load_translations locale_file }

I am using the gibberish and gibberish_translate plugins and am quite happy with those. Of course, it would be nice if they were rewritten to use the new I18n API. Another TODO item is to move over attribute names from Gibberish to I18n. I have my own override of ActiveRecord::Base.human_attribute_name that is no longer needed now that the 2.2 version of the method so nicely does I18n message lookups (with a key naming convention like 'activerecord.attributes.model_name.attribute_name').

Thanks so much to Sven Fuchs and the I18n team, Jeremy Kemper, and all the others who made I18n a part of Rails! Code will be cleaner from now on and life easier...

3 Comments

Fri September 05, 2008
Programming

Test Driven Development with Ruby

I am closing my DreamHost account, partly because they have security issues and partly because I just don't need it anymore. I had to move my Test Driven Development with Ruby article to a new home here at marklunds.com. Hopefully Google will pick up the new page and update its index soon.

Make a Comment

Wed September 03, 2008
Programming

Modularizing Your Rails App with Concerns

In his keynote here at RailsConf Europe 2008, David (DHH) talked about living with legacy code, how we should enjoy it instead of trying to avoid it, and how it can give us new insights by showing us how we have grown as developers. I loved the keynote and it resonated really well with my own experiences. It's also highly relevant with my current work at Newsdesk and Simple Signup.

As an example of refactorings you might find yourself doing in legacy Rails apps, David showed us how to break a big application helper or a fat model into Ruby modules. The idea was to find groups of methods that represent a certain concern or aspect of your app and collect those in a module. This is typically not done for reuse but to make your code more readable and easier to navigate.

Last week I found myself creating a plugin for a certain aspect of my application, namely the acceptance of terms of its service. It was a minimalist plugin with just a helper method and a controller filter method. The code was highly application specific and thus it felt wrong to keep it in the plugin directory. After all, plugins are supposed to be shared across applications and my plugin was inherently tied to the application. Still, I wanted to have the ability to keep an aspect of my application in its own directory, especially when the aspect spans across several layers of MVC. The solution I came up with was to create a new directory RAILS_ROOT/app/concerns with a sub directory and an init.rb file for each concern, very much like with plugins. I then generated a concerns plugin to make sure my concerns get loaded:

Dir[File.join(RAILS_ROOT, "app", "concerns", "*", "init.rb")].each do |init_file|
  require init_file
end

I talked to David (DHH) about this approach and the funny thing was that he had experiemented with it too. David says the approach can be appropriate if you use it wisely. If you overuse it your concerns directory will end up being a new "garbage can" and bring you back to the problem that you were trying to solve with concerns in the first place... :-)

4 Comments

Tue September 02, 2008
Programming

Introducing Rails Mentor

Me and fellow Rails developer Carl-Johan Kihlbom from Gothenburg have just founded Rails Mentor. Rails Mentor is a network of Ruby on Rails experts and we offer mentorship and training in anything related to Ruby on Rails. Me and Carl-Johan are committed to Rails best practices and together we have a broad experience of applying Ruby on Rails to varying types of projects. Now we would like to help spread this knowledge to others. If you need to be brought up to speed quickly with Rails or need a code or architecture review, please don't hesitate to get in touch with us.

We are currently at RailsConf in Berlin and we are giving free Rails Mentor t-shirts so come talk to us if you want one!

Make a Comment

Tue August 12, 2008
Programming

Installing Ruby on Rails on Ubuntu 8.04 Hardy Heron

I've compiled a set of detailed instructions on how to install Ruby on Rails on Ubuntu 8.04 Hardy Heron. The instructions are available in GitHub and they show you how to turn a clean Ubuntu 8.04 install into a production ready Ruby on Rails stack including MySQL, Nginx with fair proxy balancer, Monit, and Mongrel Cluster. There are a few additions and improvements I'd like to make:

Please help me point what else is missing or how I can improve.

I'm talking to GleSYS about the possibility of offering my production setup as an installable VPS image.

12 Comments

Wed August 06, 2008
Programming

Faster Capistrano Subversion Deployments with remote_cache

This is old news, but if you are deploying an application with Capistrano from Subversion and you find yourself waiting impatiently for the checkout to happen everytime, try adding this to deploy.rb:

set :deploy_via, :remote_cache

With this setting Capistrano will keep a Subversion checkout under shared/cached-copy, and on each deploy just do an svn update followed by a local copy of the whole tree into releases. I found this to be significantly faster than the default checkout on each deploy.

I am planning to switch to Git and GitHub now, and I am curious to see if this setting will affect deployment speed with Git. Given that a git clone in my experience is blazingly fast, maybe the remote cache won't be needed?

4 Comments

Sat August 02, 2008
Programming

Installing Telenor 3g Modem (Mobilt Bredband) on Ubuntu

As far as I know none of the turbo 3g GSM modems on the swedish market officially support Linux. However, I was able to get my Telenor modem working just fine on my Ubuntu Eee just now. I used the USB_ModeSwitch software along with swedish instructions from Hasain. Once my modem was recognized I ran sudo wvdialconf. I then edited my /etc/wvdial.conf to be:

[Dialer Defaults]
#Init1 = ATZ
Init1 = AT+CPIN=<YOUR PIN HERE>
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Modem Type = Analog Modem
Baud = 9600
New PPPD = yes
Modem = /dev/ttyUSB0
ISDN = 0
Phone = *99#
Password = peter
Username = peter

Note that you have to change the PIN above. I then ran sudo wvdial and voila - I was online! Well, online on a slow and unreliable connection that is. Oh well, I'm thinking of changing to tre.se one of these days. They supposedly have the best 3g network.

4 Comments

Sat August 02, 2008
Programming

Asus Eee + Ubuntu + Rails

On thursday I bought myself an Asus Eee 900 - a tiny and cheap Linux powered laptop that is currently selling out in stores here in Sweden. I got a lot of attention in the office with this laptop and within a day it seemed every programmer in the office had an Eee on their desk.

I installed Ubuntu Eee and this was a huge improvement over the Linux OS that Asus provides. I am ashamed to admit that I haven't used Linux on the desktop for a long time and I was totally blown away by how advanced, slick, and user friendly Ubuntu has gotten.

Obviously, my ultimate goal was to install Ruby on Rails. At first I wanted to install from source in order to get an exact version and patch level of Ruby, namely the one that is officially recommended on ruby-lang.org. However, when attempting this various libraries were missing. I found a FiveRuns article listing the packages I needed but after I had installed them I ran into an issue with the MD5 library. In the end I resorted to using the ruby-full package which gives you the old tried and tested Ruby 1.8.6 patch level 111 (without the recent security patches). Here, roughly, are the steps I went through to set up my Rails environment:

  #####################################
  # RUBY
  #####################################

  sudo apt-get install ruby-full
  which ruby
  ruby -v
  # => ruby 1.8.6 (2007-09-24 patchlevel 111) [i486-linux]
  ruby -ropenssl -rzlib -rreadline -e "puts :success" 

  #####################################
  # RUBYGEMS
  #####################################

  # Get latest stable recommended release of RubyGems from rubygems.org
  wget http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz
  tar xzf rubygems-1.2.0.tgz
  cd rubygems-1.2.0/
  sudo ruby setup.rb
  # Not sure if/why this step is necessary
  sudo ln -s /usr/bin/gem1.8 /usr/bin/gem
  which gem
  gem --version
  # => 1.2.0

  #####################################
  # MYSQL
  #####################################

  sudo aptitude install mysql-server mysql-client libmysqlclient15-dev

  #####################################
  # Some useful Gems
  #####################################

  sudo gem install rails mongrel capistrano mysql

I haven't double checked those instructions for accuracy since I would have to re-install my OS to do that. If you find errors or have improvements, please let me know. Here is a sample of articles on the subject that you might want to check out if you want to dig deeper:

3 Comments

Fri July 04, 2008
Programming

Rails Configuration: Yielding self in initializer

I've come across situations in Rails where you want to repeatedly invoke methods on some class with a long name and it gets ugly and tedious that you have to repeat the class name on every line. I just realized that unlike in the config/environments files the config object is not available in the config/initializer files. I use the AppConfig plugin to parameterize my application and I came up with the yield_self method to make my config/intializers/app_config.rb more readable:

# Method to supplement the app_config plugin. I want to crash early and be alerted if
# I have forgotten to define a parameter in an environment.
def config_param(name)
  AppConfig.param(name) do
    raise("No app_config param '#{name}' defined in environment #{RAILS_ENV}, " +
      "please define it in config/environments/#{RAILS_ENV}.rb and restart the server")
  end
end

class Object
  def yield_self
    yield self
  end
end

AppConfig::Base.yield_self do |config|
  config.site_name = "Simple Signup"
  config.admin_email = '"Simple Signup" <info@simplesignup.se>'
  config.exception_emails = %w(info@simplesignup.se)
  config.email_prefix = "[#{config.site_name}]"
  config.signup_timeout = 5
  config.send_activate_email = false

  config.transaction_fee = lambda do |price|
    price.blank? ? 0.0 : [10.0, 5.0+0.035*price.to_f].max.round(2)
  end

  config.bank_fee = lambda do |price, card|
    0.0
  end
end

ExceptionNotifier.exception_recipients = config_param(:exception_emails)
ExceptionNotifier.sender_address = %{peter_marklund@fastmail.fm}
ExceptionNotifier.email_prefix = "#{config_param(:email_prefix)} ERROR: "

The method config_param might be more appropriately named app_config. That will be a refactoring for another day though.

Make a Comment

Thu June 26, 2008
Programming

Ruby Gotcha: Default Values for the options Argument Hash

Like in Java, method arguments in Ruby are passed by reference rather than by copy. This means it's possible for the method to modify the arguments that it receives. This can happen unintentionally and be a very unpleasant surprise for the caller. A good example of this is the typical options = {} at the end of the method argument list. If you set a default value in that hash then that is a potential issue for the caller when the options hash is reused in subsequent calls (i.e. in a loop). See below for an example:

  def foreign_key(from_table, from_column, options = {})
    options[:to_table] ||= from_column.to_s[/^(.+)_id$/, 1].pluralize
    execute ["alter table #{from_table}",
            "add constraint #{foreign_key_name(from_table, from_column)}",
            "foreign key (#{from_column})",
            "references #{options[:to_table]}(id)",
            "#{on_delete_clause(options[:on_delete])}"].join(" ")
  end
  
  def foreign_keys(from_table, from_columns, options = {})
    from_columns.each do |from_column|
      foreign_key(from_table, from_column, options)
    end
  end

In the first invocation of foreign_key options[:to_table] will be set (if it isn't set already) to the destination table of the first column. The options[:to_table] value will be retained throughout the loop causing all foreign keys to point to the same table. The fix is to make to_table a local variable or to do add an "options = options.dup" line at the beginning of the method.

Lesson learned - avoid modifying the options hash and any other arguments that the caller doesn't expect will be modified.

1 Comment

Thu June 19, 2008
Programming

Rails Testing Tip: Use Global Fixtures to Avoid Fixture Mayhem

I'm in a Rails team with mixed opinions on whether to use fixtures. Therefore we have everything from RSpec specifications that use a lot of mocking/stubbing and don't touch the database, to specifications that set up their own databse data through helper methods and the specifications that I write that rely mostly on fixture data. What I have found is that when you don't use global fixtures (a setting in your test_helper.rb or spec_helper.rb file) you can run into situations where seemingly unrelated specifications/tests fail, and fail in different ways depending on if you run them in separation, through autotest, or with rake. What is going on is test data spillover/interference between tests. This can lead to very long and frustrating debugging sessions indeed. The best way to avoid this seems to be to turn on global fixtures. This will probably increase the specification run time, an issue that I partially adress by keeping the number of records in my fixture files to a minimum. Also, I prioritize test coverage and convenient access to a common set of test data over making my specifications run faster.

Make a Comment

Mon June 16, 2008
Programming

Rails Optimistic Locking - Not Worth it for Me

When I upgraded to Rails 2.1 ActiveRecord partial updates were turned on, i.e. when you save a record only those attributes that have changed are saved to the database. In theory, if you have two almost simulateneous updates, and you have a validation rule across several columns, then those updates can render the database record in an invalid state. Of course, in practice, this is very unlikely to happen. Nevertheless, I decided to turn on optimistic locking to be on the safe side. It turned out optimistic locking caused more issues than it was worth. Suppose you have an Article model that has many instances of Chapter and also that the Article uses a counter cache. Then you can run into this issue:

article = Article.first

article.chapters.create(:name => "Summary")
# => UPDATE articles SET chapters_count = chapters_count + 1,
#    lock_version = lock_version + 1 WHERE id = XXX;

article.publish_date = 1.days.from_now
article.save
# => throws ActiveRecord::StaleObjectError

I like partial updates though since they make it less likely that simulteneous updates will clobber, since you are only writing to the db what has changed. It also makes SQL statements in the log file more readable.

I'm abandoning optimistic locking for now though.

Make a Comment

Mon June 16, 2008
Programming

Ruby Gotcha: Symlinked Scripts and File.dirname(__FILE__)

If you have a Ruby script say in ~/src/ruby/my_script that you are symlinking to from ~/bin/my_script, then invoking File.dirname(__FILE__) in that script will yield the directory of the symlink not the directory of the script file. If you want the directory of the script file you can do this instead:

THIS_FILE = File.symlink?(__FILE__) ? File.readlink(__FILE__) : __FILE__

THIS_FILE will contain the path to the script file instead of the path to the symlink. This is valuable if say you want to require some Ruby library from your script and you are using a relative path.

5 Comments

Mon June 16, 2008
Programming

Rails Tip: Validating Option Arguments

I think it's a good convention to validate options arguments passed to methods, i.e. to make sure they have valid keys and values. Misspellings can otherwise lead to unnecessary debugging sessions. Rails comes with the assert_valid_keys method. I added the assert_value method:

module ActiveSupport #:nodoc:
  module CoreExtensions #:nodoc:
    module Hash #:nodoc:
      module Keys
        # Assert that option with given key is in list
        def assert_value(key, options = {})
          options.assert_valid_keys([:in])
          if !(options[:in].map(&:to_s) + ['']).include?(self[key].to_s)
            raise(ArgumentError, "Invalid value '#{self[key]}' for option '#{key}' must be one of '#{options[:in].join(', ')}'")
          end
        end
      end
    end
  end
end

Example usage and RSpec specification:

def price(feature_set_ids, options = {})
  options.assert_valid_keys([:billing_period])
  options.assert_value(:billing_period, :in => ["annual", "monthly"])
  # method logic here...
end

# Rspec:

it "cannot be invoked with an invalid option" do
  lambda {
    @campaign.price([23], :foobar => true)
  }.should raise_error(ArgumentError)
end

it "cannot be invoked with an invalid billing period" do
  lambda {
    @campaign.price([23], :billing_period => :foobar)
  }.should raise_error(ArgumentError)
end

1 Comment

Mon June 16, 2008
Programming

Rails plugin: acts_as_state_machine_hacked

I've been using the acts_as_state_machine plugin in a couple of projects. I think the syntax and functionality of the plugin is quite nice. It allows you to easily define states and events (transitions between states) for your ActiveRecord model. However, I wanted to be able to see which states are available in the current state. Also I thought that invoking an event, such as user.activate!, when the user is in a state where the activate event is not available should not yield a silent failure, but rather throw an exception. Also, if the event fails to fire because of a guard condition then an exception should also be raised. I encapsulated those changes in the plugin acts_as_state_machine_hacked.

1 Comment

Mon June 16, 2008
Programming

DreamHost Hacked: All My Files Exposed Publicly

An ex-colleague of mine discovered that all my files in my home directory at the hosting company DreamHost were publicly viewable and downloadable on the web. I was quite shocked. I had certainly not intended to share all my private files with the world, especially since they contained some highly sensitive information. I assumed my account at DreamHost had been hacked. However the response from DreamHost support was that this was not the case. They explained that it was merely a symbolic link to my home directory that had been created:

"if you would like to keep this from happening you can prevent all other users on the server from viewing your account's files by enabling the Enhanced Security feature for your user. Just go to the Users > Manage Users section of your panel, click the "Edit" link next to your user, and then check to enable the Enhanced Security option. Hit the "Save Changes" button and you should be set in about 20 minutes.

The /home/ directory is public and it is not a security breach that the other user was able to create a symbolic link to /home/. Other users on the server have always had the same access, which means that they have been able to view your files but they absolutely cannot make any changes to your files or folders. The Enhanced Security feature takes it a step further and prevents any user from even viewing your files or folders.

So, just to be clear, there is no indication of a server hack or any security intrusion."

I wrote back that I had changed to "Enhanced" security and that my files were still exposed. Here are some excerpts from their second reply:

"Ultimately this was just some funny permissions on your home directory which caused this to be allowed to happen."

"When I changed your home directory's group ownership back to your default group (pg136611) this corrected the insecurity of other user's accessing your files via apache"

"The interesting part is it may have been enabling the extra web security which caused this insecurity."

A few days later I found that my files were still exposed and I had to manually change the group of my home directory. Basically as far as I'm concerned this means the issue has still not been fixed in a reliable fashion.

I've heard no apology from DreamHost so far. In fact, there is not much in their replies that indicates that they are even taking the issue very seriously. I'm quite disappointed and I am not left with much confidence in DreamHost when it comes to security and privacy.

6 Comments

Mon May 19, 2008
Programming

Rails Testing Tip: Validate your Fixtures

I realized today how important it can be to validate the fixtures you use in your Test::Unit tests or RSpec specifications. I made some schema and validation changes and neglected to update all my fixtures which lead to a long and tedious debugging session. I added this RSpec specification to make sure I never have invalid fixtures again:

describe "fixtures" do
  it "should be valid" do
    ActiveRecord::Base.fixture_tables.each do |table_name|
      klass = table_name.to_s.classify.constantize
      klass.send(:find, :all).each do |object|
        puts("#{klass.name} #{object} is invalid: #{object.errors.full_messages.join(', ')}") if !object.valid?
        object.should be_valid
      end
    end
  end
end

Note: the fixtures_tables method is just a method I have defined that returns a list of all my fixture tables and I use it to set global fixtures in my test_helper.rb and spec_helper.rb files. If you are not using global fixtures, you can use this spec instead:

describe "fixtures" do
  it "should be valid" do
    Fixtures.create_fixtures(fixture_path, all_fixture_tables)
    all_fixture_tables.each do |table_name|
      begin
        klass = table_name.to_s.classify.constantize
        klass.send(:find, :all).each do |object|
          puts("#{klass.name} #{object} is invalid: #{object.errors.full_messages.join(', ')}") if !object.valid?
          object.should be_valid
        end
      rescue NameError
        # Probably a has and belongs to many mapping table with no ActiveRecord model
      end
    end
  end

  def fixture_path
    Spec::Runner.configuration.fixture_path
  end

  def all_fixture_tables
    Dir[File.join(fixture_path, "*.yml")].map { |file| File.basename(file[/^(.+)\.[^.]+?$/, 1]) }    
  end
end

I think it would be nice if Rails/RSpec has fixture validation built in and turned on by default.

Make a Comment

Mon May 12, 2008
Programming

Rails Tip: Validating Option Arguments in your Methods

I think it's a good convention to validate that options passed to methods have valid keys and values. Misspellings can otherwise lead to unnecessary debugging sessions. Rails comes with the Hash#assert_valid_keys method. I added the assert_value method:

module ActiveSupport #:nodoc:
  module CoreExtensions #:nodoc:
    module Hash #:nodoc:
      module Keys
        # Assert that option with given key is in list
        def assert_value(key, options = {})
          options.assert_valid_keys([:in])
          if !(options[:in].map(&:to_s) + ['']).include?(self[key].to_s)
            raise(ArgumentError, "Invalid value '#{self[key]}' for option '#{key}' must be one of '#{options[:in].join(', ')}'")
          end          
        end
      end
    end
  end
end

Example usage:

  def price(feature_set_ids, options = {})
    options.assert_valid_keys([:billing_period])
    options.assert_value(:billing_period, :in => ["annual", "monthly"])
    ...

Corresponding RSpec specifications:

    it "cannot be invoked with an invalid option" do
      lambda {
        @campaign.price([23], :foobar => true)        
      }.should raise_error(ArgumentError)
    end
    
    it "cannot be invoked with an invalid billing period" do
      lambda {
        @campaign.price([23], :billing_period => :foobar)        
      }.should raise_error(ArgumentError)      
    end

Make a Comment

Thu February 28, 2008
Programming

RSpec Presentation

I gave a presentation on RSpec today at Diino.com - an online backup provider - and the slides are available here.

1 Comment

Tue February 26, 2008
Programming

Stockholm Ruby User Group Meetup Tonight

We are having a Stockholm Ruby User Group (SHRUG) meetup tonight at Connecta here in Stockholm and it's good fun. Martin Kihlgren demoed his Grusome server based 2D game that uses a server written in Ruby and C for performance. Albert Ramstedt organized an RRobots workshop where you get to code your own little Ruby Robot that will fight other robots.

I gave the presentation "Building Web Apps with Rails 2 - Conventions and Plugins I Use" and the slides are available.

2 Comments

Tue November 27, 2007
Programming

Rails Internationalization with new gibberish_translate Plugin

We needed to internationalize the user interface of a Rails application that we are building and looked at a plethora of alternatives, such as Globalize, Globalite and the localization plugin by DHH. Finally we settled for the Gibberish plugin. Gibberish uses an unusual hybrid approach with both inline english texts in the code and keys for message lookups. A typical Gibberish lookup looks like this:

    update_show_page("The course has been booked"[:bookings_book_confirm])

To complement the Gibberish plugin I've drafted the gibberish_translate plugin that adds a script for extracting all message lookups from a Rails app and a controller with a web UI for doing translations. The plugin also keeps track of which english texts a translation was made from so that you can flag when english texts are changed. The plugin avoids using the database and works directly against the YAML message files in the lang directory.

8 Comments

Mon November 05, 2007
Programming

Rails Gotcha: Date Arithmetic, Time Zones, and Daylight Savings

Timezones and daylight savings can cause us programmers a lot of headache. This is evidenced by the fact that probably the most popular post in this blog ever is the one about a timezone aware datetime picker. Today I spent several hours debugging a problem a client was having with an eternal loop caused by the daylight saving transition October 28/29 in conjunction with 1.day.since. One of the chapters in the Code Review PDF by Geoffrey Grosenbach is about keeping time in your Rails applications in UTC. After todays exercises I must say I could not agree more with Geoffrey. In fact, I would go as far as to say that date arithmetic in Rails 1.2 is not reliable unless you set your ENV['TC'] variable.

Read More | 4 Comments

Fri November 02, 2007
Programming

Ruby on Rails 101: Presentation Slides for a Five Day Course

UPDATE: The slides have been updated for Rails 2.3 (in February 2009) and are available here.

I've decided to share the presentation slides that I developed for the five day introductory Ruby on Rails course that I held in June here in Sweden. All in all it's 340 slides available under a creative commons license. You can download the slides as a PDF file here or view them over at Slideshare. To give you an idea of what's inside, here are the chapters:

I hope the slides will be useful in helping people learn and teach Rails. I'd like to thank David Heinemeier Hanson for creating such a wonderful framework and Dave Thomas for doing such a great work documenting it.

24 Comments

Fri November 02, 2007
Programming

Rails Search Plugin Review: acts_as_fulltextable

The other day I had the opportunity to install and evaluate the recently announced acts_as_fulltextable Rails plugin. The raison d'etre of the plugin is to offer easy access to the MySQL full-text search engine. The plugin uses a FulltextRow ActiveRecord model stored in a MyISAM table with a polymorphic reference (id and model name) to the model that you want indexed and a single column to hold the search index text. All you need to do is basically add an acts_as_fulltextable declaration to the model that you want to search and this will then create a has_one :fulltext_row relationship and the appropriate callbacks to populate the index on create, update, and delete.

The acts_as_fulltextable plugin mostly exceeded my expecations, especially in terms of easy setup and maintenance. It makes site-wide search (across all models) as well as model specific search (constrain your search to one or more models) very easy. A strength of the plugin is the simplicity. There is very little code and the architecture is straight forward and easy to grasp.

Something to be aware of is that MySQL uses OR searches by default. I wanted to follow the Google convention of AND searches and found myself having to prefix all query terms with a plus sign to accomplish this. Is there a better way? A more serious issue is that MySQL doesn't seem to support word stemming. It turns out that the acts_as_fulltextable plugin does a gsub on search queries to add a trailing * to each search term. MySQL calls this truncation. The star ("*") works as a wild card. If you are aware of this feature you can use it to manually work around the absence of stemming, just search for house* and that will match both "house" and "houses". It will also match "household" which may or may not be what you want.

A documented gotcha that I ran into is that if you combine the leading plus sign and the trailing star (i.e. use both boolean AND and truncation) then stop words will not be removed from your query and thus you may end up with zero results. This is very annoying. What is the solution? I don't know. Maybe go with OR searches, live with the stemming issue (without the star), or find a way to remove stop words yourself. Stop words are supposedly configurable in MySQL.

The acts_as_fulltextable plugin was created because of stability issues with Sphinx and Ferret and because Solr was considered overkill. Lucene/Solr appears to be the most well documented, reliable, scalable, and flexible open source search engine out there. My guess is that Solr is still your best best if you are a search power user and your requirements are high. Because of its simplicity and ease of maintenance though, acts_as_fulltextable offers a pretty attractive lower end alternative.

3 Comments

Wed October 31, 2007
Programming

Rails Plugin: mysql_requirement

I added the plugin mysql_requirement that allows me to check the encoding/charset settings of the MySQL server as well as its version when my Rails application starts up and abort otherwise.

2 Comments

Wed October 31, 2007
Programming

Rails Tip: Configuration Parameters

The PeepCode Code Review PDF has some nice advice about how best to deal with configuration parameters in your Rails applications. Traditionally most of us have probably just stuck global Ruby constants in our environment.rb files, but there are more structured ways of doing it. I've started using the app_config plugin and it seems to work fine so far. To make sure I haven't forgotten to define a parameter in an environment I access my parameters via a custom config_param method:

# Method to supplement the app_config plugin. I want to crash early and be alerted if
# I have forgotten to define a parameter in an environment.
def config_param(name)
  AppConfig.param(name) do
    raise("No app_config param '#{name}' defined in environment #{RAILS_ENV}, " +
      "please define it in config/environments/#{RAILS_ENV}.rb and restart the server")
  end
end

Make a Comment

Sun October 07, 2007
Programming

RailsConf Europe 2007: Rails and the Next Generation Web

A sponsored presentation at the conference that I think stood out in how professionally it was executed and also touched on interesting topics was the one by Craig R. McClanahan from Sun. As I learned later, Craig has one of the most impressive track records I've seen in the Java community, being the creator of Struts and a contributer to a wide range of technologies including Tomcat and JavaServer Faces. In his talk Craig was urging Rails developers to move beyond the traditional three tier architecture of web applications. In plain english this means moving away from the classic scenario of a dumb browser talking to a single Rails application connected to a single database. Three developments in the industry today that are moving us away from this architecture are:

Craig was urging Rails plugin developers to avoid hard wired dependencies on ActiveRecord. Examples of plugins that succeed with this are acts_as_authenticated, make_resourceful, and paginator. Craig ended with three pieces of advice:

The presentation slides are available online.

Make a Comment

Fri October 05, 2007
Programming

Ruby on Rails Deployment on FreeBSD

I did a Ruby on Rails FreeBSD deployment for a client the other day and I thought I'd share what I learned in an instructional format here. Previously I had mostly deployed on Linux (Suse, CentOS etc.) and I was curious to see what the FreeBSD experience would be like. I googled for instructions and immediately found the RailsOnFreeBSD page in the Rails Wiki. Other than that I couldn't find much Ruby on Rails and FreeBSD specific instruction out there. Note - most of the instructions in this post are not specific to FreeBSD but are generic Ruby on Rails deployment steps for Unix.

We were migrating from Windows to FreeBSD and the goal was to eliminate single points of failure. We settled on two application servers with FreeBSD 6.2 on HP hardware, both running the web and app tiers in the vanilla Rails deployment stack, i.e. Apache 2.2.4, mod_proxy_balancer, Mongrel cluster, and Mongrel. A load balancer external to the Rails system would load balance between the two Apache servers. The database we use is MySQL 5 and it sits on a separate server. The idea is to add another db server with some form of MySQL replication. We have yet to decide which replication to use and recommendations are welcome. For deployment we use Capistrano 2.

Read More | 10 Comments

Mon October 01, 2007
Programming

Rails Migration Gotcha: Forgetting to set active_record.schema_format to :sql

If you rely on any SQL not supported by Rails migrations such as foreign keys - don't forget to set config.active_record.schema_format = :sql in your environment.rb. Otherwise your test database will be out of synch with your development database. I really would prefer if Rails used the SQL dump format for setting up the test database by default as that would be more reliable.

The reason I came across this now was actually that I found what seems to be a bug with Rails migrations. Given this create_table statement in my migration:

    create_table :users do |t|
      t.string :username, :null => false
      t.string :role, :null => false
      t.timestamps 
    end

Rails will create this statement in the dump file:

  create_table "users", :force => true do |t|
    t.string   "username",   :default => "", :null => false
    t.string   "role",       :default => "", :null => false
    t.datetime "created_at"
    t.datetime "updated_at"
  end

The difference is subtle but important to me. In the dump format Rails insists that the default value is the empty string. This is not what I want when I say that a column is not null, since with MySQL 5, even with SQL mode set to traditional, the column will then be defaulted to the empty string when someone tries to set the column to NULL.

1 Comment

Mon October 01, 2007
Programming

RSpec Tip: Keeping Controller Specs DRY

I am not yet converted to the idea of testing/specing views in separation so I usually invoke integrate_views at the top of my controller specs. I also have a bunch of helper methods that I want to reuse across my specs. To encapsulate those needs and DRY up my specs I came up with this little method that I keep in my spec/spec_helper.rb file:

# Describe a controller the way we want to do it for this app, i.e. with
# views integrated and with certain controller spec helper methods available
require File.join(File.dirname(__FILE__), "controller_spec_helper")
def describe_controller(controller, &block)
  describe controller do
    include ControllerSpecHelper
    integrate_views
    block.bind(self).call
  end
end

Read More | 1 Comment

Wed September 26, 2007
Programming

Rails Discussion: ActiveRecord vs SQL

Here is my Rails quote of the day, form a Rails mailinglist:

"When I was at the first RailsConf I was talking to someone about having used SQL for 15 years and that I was struggling with AR. At that moment someone grabbed me by the arm. I turned to look into the familiar face of Martin Fowler! He said to me, "If you know SQL that well you should just use SQL.""

1 Comment

Fri September 21, 2007
Programming

RailsConf Europe 2007 Notes: Dave Thomas Keynote

The Keynote by Dave Thomas was to me the most inspirational and profound talk at the conference. I have the greatest admiration for Dave Thomas. I can't offer Dave's entire talk in every detail here, nor do I know if that would be appropriate. What I have is key fragments from sentences in the talk presented sort of in the format of a poem. Hopefully someone who didn't hear the talk can make something out of it. Enjoy...

Read More

Fri September 21, 2007
Programming

RailsConf Europe 2007 Notes: BDD, RSpec, and Story Runner

For me, RailsConf Europe 2007 in Berlin started out with a BDD/RSpec tutorial with David Chelimsky, Dan North, and Aslak Hellesøy. In the first part of the tutorial the presenters gave an overview of the theory and background of Behaviour Driven Development (BDD). All I can offer here is an unstructured list of keywords and notes that I was able to scrabble down while listening:

  Domain Driven Design.
  Having engineers and business people speak the same language.
  SOA
  Using contracts that say: this is all I'm going to do for you
  Focusing on outcomes and reducing features  
  The Agile Alliance
  The false civil engineering analogy of building bridges
  SCRUM  
  Problem: programmers deleting tests they don't understand
  Format for user stories: index cards
  The given-when-then format
  Book: "User Stories Applied"
  All features should be traceable to a persona
  Keeping personas in FaceBook is a fun idea
  Example Driven Development  

Read More

Fri September 21, 2007
Programming

RailsConf Europe 2007

I sometimes tell people how the best decision I made in life was to get into salsa dancing. It was how I met my girlfriend Janne and it has simply been countless hours of fun and magic. It has also made my confidence and social skills grow which has helped me in other areas of life as well. Visiting RailsConf Europe 2007 has to rank right up there with one of those great decisions and it feels like a milestone. It gave me inspiration on many levels and I enjoyed it more than I've enjoyed any previous conference. What made all the difference for me was the networking. Meeting so many friends and colleagues from the past and having the opportunity of talking to thought leaders in the Rails ecosystem is just amazing. Now that three intense days of excitement is over I miss it already.

Read More | 1 Comment

Thu September 13, 2007
Programming

Rails Deployment Tip: FiveRuns.com

If you are a Rails developer you have probably heard of FiveRuns with their RM-Manage service offering, providing monitoring and statistics for your Rails apps. I've wanted to try the service for quite a while and today I finally got around to doing so on a production server at one of my clients. All I can say is you have to try it to believe it - it's a super slick, user friendly, and powerful service.

There was a minor glitch in the setup. The FiveRuns monitoring client didn't find our Rails app since it was in a non-standard location. I entered the FiveRuns Campfire chat room and got live support within minutes and quickly had a resolution. Once we installed the FiveRuns Rails plugin and restarted the server we had live Rails response time and profiling statistics within minutes. The first impression I have of the service is just great. We will be evaluating the service over the next 30 days and I'll post here any new findings we make. Barring any really big issues coming up, I consider FiveRuns to be a huge value add to the Rails ecosystem, almost like a milestone, and I will be recommending it wholeheartedly to all my clients from now on.

Make a Comment

Wed September 12, 2007
Programming

Rails Tip: Nested Layouts

This is just a feature I always wanted in Rails - nested layouts, i.e. the ability to have one master layout (application.rhtml) for your whole site, and then to have layouts within that that differ from section to section. Today I stumbled across this hack - just a single helper method that allows the use of nested layouts. Sweet. I'll re-post the code here:

module DocHelper
  # Nested layouts, see: http://fora.pragprog.com/rails-recipes/write-your-own/post/144
  def inside_layout(layout, &block)
    @template.instance_variable_set("@content_for_layout", capture(&block))

    layout = layout.include?("/") ? layout : "layouts/#{layout}" if layout
    buffer = eval("_erbout", block.binding)
    buffer.concat(@template.render_file(layout, true))
  end  
end

 

<% inside_layout 'application' do %>
  <div style="font-size: 150%">
    <%= yield %>
  </div>
<% end %>

5 Comments

Thu September 06, 2007
Programming

Rails and Transactional Saves

Correction: ActiveRecord::Base#save does use transactions in Rails by default, see Jarkkos comment.

In Rails, models are not saved within a database transaction by default. This means that if you are updating some other record in a callback like after_save then the record will still be saved even if an exception is raised in the callback. If you care a lot about data integrity this is not satisfactory. One way to remedy the problem might be to simply override the save and save! methods in your model like this:

  def save
    Post.transaction { super }
  end

  def save!
    Post.transaction { super }
  end

That's the workaround I'm going to use for now.

5 Comments

Wed September 05, 2007
Programming

Ruby as a Bash Substitute

It's really nice how Ruby can liberate me from languages that I don't like as much and am not as fluent in, such as Bash, and Perl. Today I was tearing my hair over a little bash database backup script that I needed to write until I realized I'm much better off writing it in Ruby. Bash translates amazingly easy to Ruby with the syntactic similarity, the availability of the back ticks (`), and libraries such as FileUtils. Once I switched to Ruby the backup was solved faster and with more confidence.

This experience got me thinking of the expression "when all you have is a hammer, everything looks like a nail". If I'm not mistaken the Pragmatic Programmers use not only the advice "use the right tool for the job" but also "master your tools". There is a pretty obvious contradition between those two pieces of advice. It's important to not be afraid of picking up new tools, but it's also important to not underestimate the cost involved in learning the new tools in depth. Some tools I seem to pick up and learn quickly, others I can struggle with for years without ever really feeling confident or happy with them.

Make a Comment

Fri August 31, 2007
Programming

Rails Testing: Switching from Test::Unit to RSpec, What are the Advantages?

It's been a couple of months now since I switched from Rails built in Test::Unit framework to using RSpec and I've realized that RSpec has already brought some major advantages for me. Just to clarify, I still run all my old Test::Unit tests and I still write Test::Unit integration tests to complement my specs. The Rails testing framework and RSpec can run nicely side by side through the rake command. Here is what RSpec has given me:

Another realization that has come to me lately, is that yes, regardless of what code you are writing TDD/BDD is a good idea. However, if you are writing any kind of API or framework that is to be reused across several applications, tests/specs and well defined interfaces are almost a necessity. If you don't have them, you might be better off opting for no reuse. Reuse is the holy grail of software development, and it's hard.

1 Comment

Fri August 31, 2007
Programming

Rails Gotcha: The Contract of 1.month.ago

At first glance the contract of the beautiful 1.month.ago construct in Rails is maybe obvious. However, what should it return if the current time is the 31st of March? Remember that february only has 28 days. Well, according to Rails 1.2.3 the answer is 1st of March, but according to Edge Rails the answer is (and I agree with Edge Rails here) 28:th of February.

The 1.month.ago issue is something that bit me and that you should be aware of. So with Rails 1.2.3, if you want to be sure to get the beginning of the previous month, you need to say 1.month.until(Time.now.beginning_of_month). At least that's the workaround I'm using now.

2 Comments

Fri August 31, 2007
Programming

Rails Deployment: Rapid Setup with the Machinify Gem

I used the Machinify Gem by Bradley Taylor (Mr RailsMachine) the other day to install a Rails stack on a piece of Ubuntu 7 Xen VPS and I must say it worked really nicely. The Gem uses rake to install the common stack of MySQL 5, Apache 2 with mod proxy load balancer, Mongrel Cluster, and Mongrel. The gem also takes care of installing all the obscure Debian packages needed that you don't even want to know about.

There is a small glitch currently in the Machinify gem in that the Mongrel Cluster version is hardcoded in its mongrel.rake file. Once I corrected that and ran rake stack:install again it worked like a charm. Related to this issue, I was trying to find out how to fetch the current version of a Gem and the best I could come up with was:

# On the command line:
gemwhich capistrano
=> /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.0.0/lib/capistrano.rb

# In Ruby:
`gemwhich capistrano`[%r{-([0-9.]+)/lib/[^/]+$}, 1]
=> "2.0.0"

If you are on Cent OS you should check out RubyWorks by ThoughtWorks (the installer, not the singer).

Make a Comment

Tue August 28, 2007
Programming

RSpec Gotcha: Incompatibility with Engines 1.2.0

If you are trying to get RSpec to work in a Rails application that uses Engines you may find that RSpec errors out when you try to run it. It seems the reason is related to the file testing.rb in the engines plugin. If you comment out the line that requires that file (in vendor/plugins/engines/init.rb) you will have worked around the issue:

#require "engines/testing" if RAILS_ENV == "test"

Make a Comment

Sun August 12, 2007
Programming

Rails Deployment: A Little Checklist

After having fought myself through a couple of Ruby on Rails deployments, it seems there are a lot of little annoying things you need to remember. Some of them are not super critical, at least not in the short run, but they can grow into bigger problems over time. Here is the list:

1 Comment

Sun August 12, 2007
Programming

Rails Deployment: Dealing with 404s

You probably use Jamis Buck's Exception Notifier plugin to be notified by email of exceptions thrown on your production server. Typically though you don't want to be notified of 404s, at least not when they are caused by an RSS reader requesting an RSS feed that no longer exists, or some spam robot or search engine fetching URLs fetching URLs no longer supported. When the Exception Notifier plugin catches an exception and it is one of ActiveRecord::RecordNotFound, ActionController::UnknownController, or ActionController::UnknownAction, it figures that it is a 404. By default then no email is sent and instead the method render_404 is invoked which in turn renders public/404.html. This is quite appropriate. Unofortunately though you can't trace in the log what the request looked like that caused the 404. The other limitation is that if no route matches a n incoming request, then an ActionController::RoutingError is thrown, yielding a 500 and an exception email. To get consistency in how 404s are dealt with, i.e. make sure they are always fully logged and tracable, but never cause email notifications, I override the render_404 method from the Exception Notifier plugin in my ApplicationController:

  # Overriding the version in ExceptionNotifiable so that we can log the environment and figure
  # out where the request is coming from.
  def render_404
    logger.error("render_404 invoked for request_uri=#{request.request_uri} and env=#{request.env.inspect}")
    super
  end

I then add a catch all route at the end of my routes.rb file:

  map.connect '*anything', :controller => 'application', :action => 'render_404'

4 Comments

Sat August 11, 2007
Programming

Rails Deployment: Looking up User ID from a Session ID

Suppose you get an exception notification and you want to know which user was browsing the site when the exception occured. The exception email has the HTTP_COOKIE header with the _session_id. Assuming you are using ActiveRecordStore and that you store the user ID in the session, you can look it up with an incantation like the following:

CGI::Session::ActiveRecordStore::Session.find_by_session_id('c5934a07b12ae9d90cf6be4e7d48b361').data

I keep forgetting the module path to the Session class so I thought I'd post this note. The corresponding Rails file is active_record_store.rb in ActionPack.

1 Comment

Wed August 01, 2007
Programming

Rails Testing: Quoting angle brackets in assert_select

This is just a little detail but today I rediscovered the way to quote angle brackets in argument values in HTML::Selector expressions in your assert_select commands (and corresponding response.should have_tag commands in RSpec). I tried using back slashes before I realized that I needed to enclose the argument value in single quotes. Notice the nested angle brackets ([ and ] signs) below:

  it "Should have the allow_public_messages radio buttons on the edit contact info page" do
    get :edit_my_options
    response.should be_success
    response.should have_tag("input[name='profile[allow_public_messages]']")
  end

Make a Comment

Thu July 26, 2007
Programming

RSpec RAILS_ENV Gotcha

I've had issues with RSpec running in both development and production environment which certainly isn't what I want and it seems others have had the same issue. The resolution is simple, force RAILS_ENV to be set to "test" in your spec/spec_helper.rb file.

1 Comment

Mon July 23, 2007
Programming

Teaching the First Rails Course in Sweden

In the last week of June I had the privilege of teaching the 5 day Ruby on Rails course I Go Ruby at the wonderful Ulvhäll Herrgård in Strängnäs, just a one hour train ride west of Stockholm. To the best of my knowledge it was the first Ruby on Rails course held in Sweden, and this of course made us feel special. The relaxed location and format of the course - a summer camp with a small group of students - also worked well to create a pleasant atmosphere.

I have previously taught evening courses in Ruby on Rails at companies like Great Works and Connecta, but this was the first time that I got to teach a longer course. We had a small group of eight students with very different experience levels and this of course posed quite a challenge for me. I found that my exercises were a tad too difficult and also that I was possibly covering too much material, at least for the less experienced students.

I developed all of the course material myself - over 300 slides plus exercises - a major undertaking consuming about a month of full time work. The basic contents of the course roughly corresponds to the Agile Web Development with Rails book, but important additions were made in areas such as Ruby, testing, and Rails 2.0.

There are tentative plans to make the course material available online at an affordable price. I am also hoping to offer two day Rails courses in Stockholm during the autumn. Eventually I'd like to offer a two day introductory course, and then one or more advanced one or two day courses on advanced topics. Stay tuned for announcements on this, and let me know if you are interested.

2 Comments

Thu May 24, 2007
Programming

Rails Testing: The form_test_helper Plugin

I've been using the form_test_helper plugin in my integration tests lately and I think it's a great way to increase test coverage of your views. The form_test_helper uses the assert_select CSS-like selectors and makes it easy to click links and submit forms and thus simulate something that comes pretty close to manual testing in the browser, unless of course you need to test AJAX functionality, in which case you need to look at Watir or Selenium. Here is some sample code:

Read More | 1 Comment

Wed May 23, 2007
Programming

Rails + RSpec: First Impressions

I started using RSpec on a client project yesterday and I've been pleasantly surprised by how easy it is to get started with and how nicely it plays with Rails. I now have a handful of specs under the spec directory and they get run by rake automatically right after my tests. I can also use rake spec:doc to generate the documentation of my specs:

Read More | 1 Comment

Tue May 01, 2007
Programming

Rails Testing: Checking for Broken URLs in Links, Images, Forms, and Redirects

I've added the ability to check for broken URLs in links, images, forms, and redirects to my Http Test plugin. How does this work? Well in your controller and integration tests an ApplicationController after filter is used to extract all the URLs on the page. For each URL we make sure it can be resolved to a route, a controller, and an action and/or template. URLs to files under the public directory will be recognized. The same check is made for redirects.

I haven't used this URL checking in production yet, but I'm hoping it will help me prevent mistyped URLs and params.

Make a Comment

Tue May 01, 2007
Programming

Bug Tracking can be Bug Prevention

I've been deploying a couple of Rails applications for a client lately and things have been running surprisingly smoothly and there have hardly been any issues at all. In one of the more complex applications though a few bugs slipped through my testing net and were caught by the excellent Exception Notifier plugin by Jamis Buck. Now, the question is, how do we typically deal with bugs as programmers? If the bug is in production we rush to fix it, push the fix out, and hope not too many people noticed it, and then try to forget about the whole thing. What we should be doing though is ask ourselves why the bug occured in the first place, how we can categorize the bug, and what kind of process and/or tests can we put in place to prevent similar kind of bugs from happening in the future. Bug tracking is a gold mine for figuring out how we can raise the quality of our software and reduce the bug rate in the future. This may be common sense, but how many of us actually use this opportunity?

1 Comment

Wed March 28, 2007
Programming

Rails Gotcha: No ordering of ActiveRecord after_save callbacks

I am setting up tagging and searching for a demo application and I am combining the plugins acts_as_taggable_on_steroids and acts_as_ferret. I was using the following setup for my users:

  acts_as_ferret :fields => [:login, :email, :bio, :tags_string]

  def tags_string
    tags.map(&:name).join(" ")
  end

However, that didn't work. When making a change to my tags I had to save the user twice for the search index to be updated. Why? Because the acts_as_taggable_on_steroid after_save callback (save_tags) runs after the acts_as_ferret after_save callback (ferret_update). Maybe the callbacks are invoked alphabetically, I don't know. Anyway, there is no explicit ordering and no after_after_save callback...

The workaround in this case was easy because of the virtual attribute tag_list that gets set before the save:

    def tags_string
      Tag.parse(tag_list).join(" ")
    end

Now we are always using the latest tag list when we update the Ferret index. Problem solved.

1 Comment

Mon March 26, 2007
Programming

Rails Extension: ActiveRecord::Base.find(:last)

I'm at the Big Nerd Ranch Ruby on Rails bootcamp in a monastery outside Frankfurt right now having a great time. One of the students asked if you can say ActiveRecord::Base.find(:last), which you can't. The value of the first argument is typically :first, but you can't use :last. Just for educational purposes I decided to change that (it's probably not something I would use in production):

ActiveRecord::Base # Make sure class is loaded, which it probably is by now anyway
module ActiveRecord
  class Base
    def self.find_with_last(*args)
      if args.first == :last
        options = extract_options_from_args!(args)
        find_without_last(:first, options.merge(:order => "#{primary_key} DESC"))
      else
        find_without_last(*args)
      end
    end

    class << self # Needed because we are redefining a class method
      alias_method_chain :find, :last
    end    
  end
end

I guess most interestingly this code illustrates how to alias a static method.

5 Comments

Mon March 26, 2007
Programming

Abandoning Textilize for Comment Formating

I've been using the textilize helper for comment formating in this blog. The textilize helper uses the RedCloth gem, which supports both Textile and Markdown syntax and it's pretty complex code with lots of hairy regular expressions. I noticed that the textilize helper will let arbitrary HTML tags through so it can mess up the HTML of a page. Seems like a vulnerability, and it's even more a problem for me since I like to be able to HTML validate my pages. I decided to stop using textilize for comments and instead use the following simpler and tighter formating:

  # HTML quote, convert line breaks to <br />, and convert URLs to links
  def simple_link_format(text)
    # From http://snippets.dzone.com/posts/show/3654
    url_regex = /(http|https):\/\/[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,5}(([0-9]{1,5})?\/.*)?/ix
    simple_format(h(text).gsub(url_regex, '<a href="\0">\0</a>'))
  end

1 Comment

Mon March 26, 2007
Programming

Capistrano is not just for deployment

I'm appreciating Capistrano more and more. It has such an elegant and powerful API, and you can use it for pretty much any commands you need to execute on one or more of your servers. Here is a simple example. For the development of this weblog I found myself quite often wanting to copy the production data to my development server. Of course at first I was doing this on the command line. The next natural step would have been to write a bash script, but these days I tend to write most of my scripts in Ruby. Rather than stick a small Ruby script under the Rails script directory I'm starting to create Rake tasks under lib/tasks though. One advantage of having scripts as Rake tasks is that you can cagalog them nicely with "rake --tasks". Here is the Rake task:

Read More | 1 Comment

Sun March 25, 2007
Programming

Rails Deployment: Running Tests in Production with Capistrano

Murphy teaches us that if something can go wrong it will. Given all the differences there are between my development and production server - operating system, web server, database, gems etc. - I can't really feel comfortable deploying my application unless I have first run my test suite not only in development but also in production. Thanks to the beautiful and powerful API of Capistrano this is a walk in the park to accomplish:

Read More | 2 Comments

Sun March 25, 2007
Programming

Rails Deployment: Check MySQL Charset on Startup

The UTF-8 support got a lot better with the Rails 1.2 release but since Rails has evolved so fast many of the instructions you'll find on the web are dated. Rails will now default the Content-Type of its responses to UTF-8 so you don't need to set that yourself in an after filter.

When it comes to configuring MySQL to use UTF-8 there are at least three pieces to keep track of - the database encoding, the client encoding, and the connection encoding. For my project it was enough to set "default-character-set=utf8" in the MySQL my.cnf config file (takes care of the database part) and set "encoding: utf8" in database.yml (takes care of the connection and client).

Read More | 1 Comment

Sat March 24, 2007
Programming

Rails Testing: Making assert_select XML safe

I've been using the new assert_select command extensively in my tests lately and it's a wonderfully powerful tool - a huge improvement over assert_tag. The only annoyance has been the warning messages "ignoring attempt to close form with link" that have been polluting my test output. The other day I decided to track the source of the messages and I discovered that others had already complained about them. I found that the root cause of the problem lies in the HTML::Document and the HTML::Tag classes that ship with Rails. More specifically the problem is in the HTML::Tag#childless? method that auto-closes HTML tags such as img, br, and hr.

Read More | 2 Comments

Tue March 20, 2007
Programming

Rails Tip: Declare your Gem versions

Many Rails applications have external dependencies in the form of RubyGem libraries. The problem with such external dependencies is that you need to make sure that you have the right gem versions installed across all your servers, i.e. development, staging, and production. The probably easiest way around this is to freeze gems into your source tree. For rails you can use the Rake task "rails:freeze:gems" and you can freeze other gems by using Rick Olsen's Gems plugin.

At the end of the day there may still be external gems that you depend on, maybe because they need to be compiled, or maybe because you want to avoid your application source tree growing too big. You can fix the version of Rails using the RAILS_GEM_VERSION constant in environment.rb. For other gems you can fix the versions by adding statements like the following to your environment.rb:

gem "RedCloth", "3.0.4"

Read More | 1 Comment

Tue March 20, 2007
Programming

RubyGems: LoadError in nested require no longer causes silent failure with Ruby 1.8.5

I had a Rails application where I was trying to reference a helper that didn't exist. What happens in Rails when you try to use a class that you haven't required is that the Ruby callback method const_missing is invoked. Behind the scenes Rails will then try to require the class for you. What was happening was that Rails was trying to require the helper file, but the helper file didn't exist, and so that failed. The problem wasn't so much that it failed, but that it failed silently - there was no error message to be seen anywhere.

Read More

Fri March 02, 2007
Programming

Rails Gotcha: ActiveRecord::Base#update_attribute skips validation

It's something others have been bitten by. Although we should know better (why didn't we read the API docs carefully?) we naivley thought we could safely update a single attribute with the update_attribute method. When you invoke my_precious_object.update_attribute(:important_attribute, erroneous_value) then Rails will happily skip validations and save all attributes to the database. This is unlike the sister method update_attributes which updates several attributes and does use validation. I'm sure validation is skipped for a reason but I'm not sure what that reason is. In particular I'm not sure if that reason is strong enough to warrant the inconsistency and the risk of skipping validations.

Read More | 10 Comments

Wed January 31, 2007
Programming

Swedish Rails Article and Course

I was very pleased to see my Ruby on Rails article published yesterday by the IT education company Informator here in Stockholm. The article is in swedish and has a title along the lines of "Rails Brings Back the Joy of Working". It was published in the slick offline paper called "Format" that Informator distributes and is also available online.

I there is enough interest (fingers crossed) I will be giving the Ruby on Rails course for Informator here in Stockholm later this year. Needless to say, I'm very excited about that opportunity!

4 Comments

Sat January 06, 2007
Programming

Rails Tip: Use the Unit Tests as Documentation

If you are looking for authoritative answers on how to use the Rails API, look no further than the Rails unit tests. Those tests provide excellent documentation on what's supported by the API and what isn't, how the API is intended to be used, the kind of use cases and domain specific problems that drove the API, and also what API usage is most likely to work in future versions of Rails.

Let me give you a concrete example. In my code I had the equivalent of a Tag class connected through a Tagging join model with two different classes, say Post and Comment. What I was trying to do was say in the Tag class something like "has_many :posts, :through => :taggings, :source => :taggable" or maybe "has_many :taggables, :through => :taggings". However, Rails refused to let me do this and kept throwing an ActiveRecord::HasManyThroughAssociationPolymorphicError. At this point I wasn't sure if I was misusing the API or if I had come across a Rails bug (although I knew that the former was more likely).

I turned first to the most authoritative documentation such as the API doc, the Agile Web Development with Rails book, and the Rails Recipes book (Recipe 22: "May-to-Many Relationships with Extra Data", and Recipe 23: "Polymorphic Associations - has_many :whatevers"). However, none of that documentation seemed to address my specific problem. Through Google I found the wiki page HowToUsePolymorphicAssociations which did address my problem. However, the solution there seemed clumsy, and I didn't know how authoritative and up-to-date it was.

It's usually a good idea to debug a problem by following the stack trace into the Rails code and try to figure out what's going on right there. However, in this particular case I found that reading the clean but complex ActiveRecord code didn't give me any easy answers. It then hit me that what I really should be doing was read the ActiveRecord unit tests. Luckily for us application developers David and the rest of the core team has provided us with a nice suite of unit tests. Once I looked at the join model test case the whole picture cleared up - the fog lifted and there was bright sunlight. Laid out right there in front of me were excellent and authoritative examples of how to use ActiveRecord associations with classes such as Tag, Tagging, Post, Comment, Category, and Categorization. I also found the particular assertion that said that the ActiveRecord::HasManyThroughAssociationPolymorphicError should be raised in my case...

One could speculate that Rails really should provide a way to do has_many through a polymorphic association, and surely there is a way to hack Rails into doing something like that. What's important though is that now I know that the approach is not officially supported or intended and that's fine with me. I like to stay on the unit tested golden Rails path and I recommend you to do the same.

1 Comment

Thu January 04, 2007
Programming

Good and Bad Programmer Practices

As programmers we have a tendency to take shortcuts that come back and bite us eventually. It is very popular to write overly complex system that seem sophisticated and make us feel smart and special. The irony is that a lot of times we think our code is so simple and obvious that it doesn't need automated tests. Here is a list of programming practices to consider:

In summary then, keep your code clean, simple, conventional, and well tested and listen to your users. Those recommendations may sound obvious, in practice though they often contradict our instincts as programmers and require discipline and courage to stick to.

Make a Comment

Wed December 27, 2006
Programming

New Rails Plugin for HTML Validation: Html Test

Update on 2009-04-16: The source code has been moved from Google Code to GitHub

Update on 2007-12-12: I have renamed the plugin from "Http Test" to "Html Test" and moved it to a new home at Google Code. Sorry about the inconvenience that this may cause those of you who have the plugin installed and want to update it. The change is essentially a namespace change from Http:: to Html:: so it should be manageable. Thanks.

As I wrote earlier I like to monitor my production sites with HTTP tests that validate their HTML and check for broken links and images. I think that serves as a good baseline check, a smoke test if you like, that the site is not fundamentally broken.

It used to be that my HTTP tests were implemented in Perl using WWW::Mechanize. Now that we have the wonderful Ruby and Ruby on Rails at our fingertips that felt like a real shame. I decided to implement my HTTP tests in Ruby and once I had gotten that far, going the extra mile to package my scripts as a Rails plugin seemed natural. The plugin is called http_test and it's available at Google Code.

Read More | 7 Comments

Tue December 12, 2006
Programming

Hype and the Rise and Fall of Programming Languages

Steve Yegge has written an article on Ruby as a competitor to Python, the Rails hype, and the rise and fall of programming languages. The article touched me deeply. Thanks to Olle Jonsson for sending the link. The article is almost a year old but should still be relevant. I especially like the last section on Ruby:

"The worldwide Ruby culture is the warmest and friendliest I've seen in my long history with programming languages. And Ruby is a sweet language. Other people seem to agree, and are taking steps to market it, which is getting them labeled as "hyper-enthusiasts" by the Sour Grapes camp. It appears to me that Ruby is doing what I wanted Python to do a few years ago, so I've finally learned Ruby and have switched most of my development over to it.

Read More

Fri December 08, 2006
Programming

MySQL Gotchas and a Deadly Sin

Coming from mostly a PostgreSQL and Oracle background I experienced some major friction working with MySQL today.

The first obstacle to overcome was the syntax for foreign keys. I started out using

  create table test (
    service_id integer NOT NULL references services(id) on delete cascade
  );

Read More | 3 Comments

Sun December 03, 2006
Programming

Usability on the Web and Embracing Simplicity

I just booked flight tickets for the christmas holiday and it got me thinking of usability. People depend on websites so much these days and it's a shame to let them suffer from poor usability. Of course, usability has improved quite a bit over the last couple of years. Indeed SEB finally uses a date widget, and I think even suggests a sensible default (the next working day), when you do payments through their online bank. For the longest time they left the user with a bare bones YYYYMMDD text input widget.

Booking tickets on the web is often a series of frustrations for me. This is in spite of the fact that I am a skilled web user. For example a form won't submit and the validation error is hardly visible. The user is left wondering if the form submitted at all, if there was a validation error, and if so, where?

Read More | 1 Comment

Thu November 16, 2006
Programming

Teaching Rails at Great Works

I've started teaching Ruby on Rails at Great Works and I'm very excited about it! It's a three evening course given to a small group of up to 10 people. We use a lot of hands on excercise rather than lecturing and the goal is to build a small Rails application. The course participants are designers rather than programmers so we needed to lay down some object orientation and programming basics before we could take on Rails. Despite of this obstacle we were able to make a lot of progress by using Locomotive for installation and scaffolding for UI generation. You can check out the slides here.

I genuinely enjoy teaching and am now looking for more opportunities to help companies learn and get started with Rails quickly. Get in touch if this sounds interesting.

Make a Comment

Wed November 15, 2006
Programming

Stockholm Rails Meetup Tonight was Awesome

We had a fantastic turnout and great atmosphere at the Rails Meetup at Valtech here in Stockholm tonight. There was a great presentation by Ola Bini on web services and Rails, and one by Christian and Albert at Adocca about caching and a bunch of plugins that they have realeased at RubyForge. All very exciting.

I gave a presentation on the Media Contact Manager system that I have built for Green Media Toolshed.

Make a Comment

Tue November 14, 2006
Programming

Rails: File Uploading and Storage

File uploading and storage is no big deal, right? I mean with Ruby it can be as easy as:

  def create
    File.open("/Users/db/_dev/rails/videos/swing.mov", "w") do |f|
      f.write(@params["video"].read)
    end
  end

Don't forget to set :multipart => true on your form though. The easiest (and most naive?) approach to uploading files is probably to use the File Column plugin. The Rails Wiki has a good page that discusses file uploads in more depth. The file system base storage approach that I've used in my Rails app is similar to what's described here. I use the after_save and after_destroy callbacks to create and delete the file in the file system - an approach also described in this Rails manual. Johan Sorensen has some code for doing buffered file uploads that might be useful for huge files.

This page is quite impressive. It has everything you ever wanted to know about the input type="file" form widget, including why browsers will ignore the value attribute if you try to set it. An annoyance is that if the user gets validation errors on a form containing a file_field widget she will have to re-select the file to upload.

The classic controversy is whether to store files in the database or in the filesystem. If you store them in the database you have a more unified and consistent storage solution. You keep all your data in one place, in one format, and the only data you need to back up is your database. On the other hand, putting files in the file system could potentially offload your database and make the database dumps smaller. Also, databases were not really intended to store files, that's what file systems are for. Personally I'm ambivalent on this issue, but due to some incidents with file system storage recently I'm now leaning more in favor of storing files in the database.

2 Comments

Sat November 11, 2006
Programming

Rails script/console - Did you know it could do all this?

Yesterday someone pointed out some features of the Rails console that I had missed out on. Here are some objects and commands to be aware of:

4 Comments

Thu November 09, 2006
Programming

Rails Patch: Rake :test should not fail silently

Swallowing exceptions and failing silently has to be one of the worst sins you can commit as a programmer and I was surprised to run into that issue with the Rake :test task. The Rake :test task (the default task) will in turn invoke the tasks test:units, test:functionals, and test:integration. Any exceptions raised within these sub tasks are not exposed directly but are merely noted as an overall test failure - obscuring the root cause of the failure. This is obviously not very debugging friendly if you have an exception raised somewhere deep down in the Rake task dependency hierarchy. I submitted a patch to remedy the problem.

3 Comments

Thu November 09, 2006
Programming

Rails QA: Watch Out for Duplication in your Fixtures

I had a long debugging session due to duplicated record keys in one of my fixture files. It turns out the YAML parser will not complain if you have duplicated record keys (the keys on the top level) or duplicated column keys. Eventhough the YAML specification says that map keys should be unique the YAML parser will happily overwrite any existing value for a key if it encounters that key again.

Under the motto of crashing early and avoiding silent failures I came up with the following unit test:

Read More | 2 Comments

Thu August 24, 2006
Programming

Rails Recipe: Checking for Broken Links in Integration Tests

Having a way to automatically check for broken links in your application can be quite valuable and there is a simple way to do that with integration tests. Here is the DSL methods that I used for that purpose along with some sample usage:

Read More | 3 Comments

Wed August 23, 2006
Programming

Rails Workaround: Preserving Nested Hash Params With List Values Across Requests

Suppose you have an HTML search form with a multi-valued select with the name "person[personal][interests][]". The trailing brackets are there to indicate to rails that this HTTP parameter should be converted to a Ruby list. Now, suppose the form submits to the search action in the contacts controller. Our params hash will then contain:

    params = {
      :controller => 'contacts',
      :action => 'search',
      :person => {
        :personal => {
          :interests => ['music', 'chess']
        }
      }
    }

Read More | 6 Comments

Fri August 18, 2006
Programming

Rails Recipe: HTML Validation

In this howto I'll show a simple approach to HTML validation that I use in my current Rails application. For me, HTML validation is a way to achieve wide browser compatibility, and to do a baseline check for correct rendering and UI brokenness. By putting ampersands and less-than signs in my test fixtures I can use HTML validation tests to check that I haven't forgotten any HTML quoting in my templates, i.e. that I haven't forgotten to use the following constructs:

<%=h some_variable %>
<%= link_to h(some_variable) ... %>

Read More | 7 Comments

Thu August 17, 2006
Programming

Rails Custom Validation

Suppose you have validation code for a certain type of data that you want to reuse across your ActiveRecord models. What is the best approach to doing this in Rails? As you probably know, Rails comes with a number of validation macros such as validates_presence_of, validates_format_of, and validates_uniqueness_of etc. One approach is to write your own validation macro that wraps one of the Rails validation macros. The validates_as_email plugin shows us how:

module ActiveRecord
  module Validations
    module ClassMethods
      def validates_as_email(*attr_names)
        configuration = {
          :message   => 'is an invalid email',
          :with      => RFC822::EmailAddress,
          :allow_nil => true }
        configuration.update(attr_names.pop) if attr_names.last.is_a?(Hash)

        validates_format_of attr_names, configuration
      end
    end
  end
end

Read More | 18 Comments

Wed August 16, 2006
Programming

Rails Recipe: A Timezone Aware Datetime Picker

NOTE! Only follow this recipe if you are using an earlier version of Rails than 2.1. As of Rails 2.1 timezone support is built into Rails in a way that makes it much easier to create timezone aware applications.

Quite often in web applications we display dates and times and also have users input them and store them in our databases. If your users are spread across the globe you really want to be able display times in the users own timezone. How do we accomplish this with Rails?

First off, as Scott Baron has pointed out, you want to grab a copy of the TZInfo Ruby timezone library. We need TZInfo since it can deal with daylight savings time (summer/winter hourly adjustments), something that the Timezone library that ships with Rails is not able to do. Installation is as simple as downloading the latest tgz file from RubyForge, extracting it into vendor/tzinfo, and requiring the library from config/environment.rb. We also add some new settings:

Read More | 124 Comments

Wed August 16, 2006
Programming

Rails Recipe: CSV Export

A common requirement from customers is the ability to export tabular data to a CSV file that can be imported into Excel. Ruby on Rails uses the standard Ruby CSV library to import test fixtures that are in CSV format. However, there is a runner up CSV library called FasterCSV which as the name suggests is faster and also has a much cleaner API. I was hesitant to use FasterCSV in production since it is still in Beta but after finding the discussion about including FasterCSV in the Ruby standard library I decided to give it a try.

Read More | 15 Comments

Thu June 08, 2006
Programming

Rails Code Generation Considered Useful

I just checked out Ryan Daigle's excellent What's New in Rails blog. There are a lot of great findings there for anyone like me who hasn't monitored closely the changes to the Ruby on Rails source code. In the post about the observer generator Ryan writes:

"On a side note, I feel like the generator utility has been getting a bad rap recently from people that frown upon auto-generation of code. Fair enough, but think of them more as best practices templates and learning tools rather than some sort of coding robot. They're really great tools for seeing how things should be done, and how things are expected to be done. If you don't want to use them, don't - but if you're not quite sure how to go about creating an observer, or controller, or model they're of great help."

Ryan puts it very well and I couldn't agree more. I've found the scaffold generator very useful in the project I'm working on now, and it's not for prototyping I'm using it, but for providing a skeleton for the production code I'm going to write.

Another advantage of the code generation in Rails is that if you use it your code will probably follow conventions (naming of controller actions for example) more closely and this will make the code easier to read and maintain for other developers. The generated code also contains best practices (such as the verify :method => :post for certain actions in controllers) that an average Rails programmer (or even a good Rails programmer) may not know about or may forget to use otherwise.

Make a Comment

Wed May 10, 2006
Programming

Presentations from the Rails Recipes Meetup

The Rails Recipes Meetup yesterday was a great success with over 20 participants and four interesting presentations. I would like to thank Xlent for hosting us and all the people who showed up and contributed to making the evening so nice.

If you are interested in coming to the next meetup (scheduled to happen in June or July), please signup for the swedish Rails mailing list at rails.se. Please also suggest presentation topics on that mailing list.

Here are two of the presentations from the meetup:

See you at the next meetup!

1 Comment

Sat April 29, 2006
Programming

Ruby on Rails Tips and Tricks

I've started a wiki page with some Ruby on Rails tips and tricks that I pick up from various weblogs and books. Hope it might be useful to others as well.

Make a Comment

Wed April 26, 2006
Programming

Starting a Startup - Hard but Doable

Here are some encouraging words from Paul Graham's How to Start a Startup article (which will make you scroll through eternity):

"You need three things to create a successful startup: to start with good people, to make something customers actually want, and to spend as little money as possible. Most startups that fail do it because they fail at one of these. A startup that does all three will probably succeed.

And that's kind of exciting, when you think about it, because all three are doable. Hard, but doable. And since a startup that succeeds ordinarily makes its founders rich, that implies getting rich is doable too. Hard, but doable."

Interesting to see the Web 2.0 Startup Toplist as well for a list of successful startups.

From Paul Graham's Y Combinator:

"Y Combinator relies on certain premises: that open-source software and falling hardware prices means that tech start-ups are cheap to finance; that large companies are no longer at the forefront of innovation; and that mature technology companies find it cheaper to buy than to build."

Make a Comment

Wed April 26, 2006
Programming

Rails Recipes Meetup in Stockholm

Each presentation is up to 20 minutes long followed by a 10 minute discussion. The presentations should be in the Rails tradition of "show don't tell". We prefer short presentations with code samples from recipes applied to real problems over the traditional type lengthy and theoretical Powerpoint presentations.

"Now, I know Rails well enough that over time I may have stumbled on some of the solutions, but that's not the point. With a collection of great recipes at hand - In-Place Form Editing, Live Search, RJS Templates, Polymorphic Associations, Tagging, Syndicate Your Site with RSS, and Sending Email with Attachments to name a few - I can quickly give my app the functionality today's web users have come to expect, then move on to the features that really set it apart from the crowd."

Make a Comment

Fri April 07, 2006
Programming

Quote of the Day

Thanks Jarkko for highlighting the following nice quote from the article Two more reasons why so many tech docs suck which argues the case of getting your users excited and using use cases for technical documentation:

One of the best techniques for creating and keeping interest is to make non-reference docs use-case driven. That way, each topic is framed by a context that matches something the user really wants to do. That way, each little sub-topic shows up just-in-time, instead of appearing to be there just-in-case.

It makes a lot of sense to me to organize technical documentation by the readers goals, i.e. to use the howto or cookbook approach. The Perl Cookbook is a great example of how this approach can be successful. In that instance the use-cases are on a very low level such as "How do I iterate over an array" but the approach should also work on a higher level, i.e. "How do I setup a weblog". High level use-cases like that are probably best broken down into smaller use-cases.

Make a Comment

Fri March 31, 2006
Programming

Quote of the Day

I was catching up with a lot of Rails related posts in my RSS reader and found DHH's bashing of James McGovern which was pretty entertaining. I also like this quote in the comments:

First they ignore you
Then they laugh at you
Then they compete with you
Then you win

I also learned today that Dreamhost upgraded to Rails 1.1. Apparently my homepage survived that upgrade. I still don't have Rails in the vendor directory but I do have web tests set up so I don't have to worry too much.

Make a Comment

Sun January 01, 2006
Programming

Lessons from Adam Bosworth

Adam Bosworth's article Learning from THE WEB is well worth reading (thanks Branimir for pointing me to it). A surprising number of the lessons have to do with simplicity, for example the KISS one:

"KISS. Keep it (the design) simple and stupid. Complex systems tend to fail. They are hard to tune. They tend not to scale as well. They require smarter people to keep the wheels on the road. In short, they are a pain in the you-know-what. Conversely, simple systems tend to be easy to tune and debug and tend to fail less and scale better and are usually easier to operate. This isn't news. As I've argued before, spreadsheets and SQL and PHP all succeeded precisely because they are simple and stupid - and forgiving."

Other lessons such as "It is acceptable to be stale much of the time" and "The wisdom of crowds works amazingly well" ring very true to me. His critique of database vendors inability to learn from the web is thought provoking.

1 Comment

Sat December 17, 2005
Programming

How I Test my Website

Introduction

This is an article that talks about the motivation for testing and how I went about writing automated tests for my personal website. It is best suited for someone with at least some familiarity with Rails but who hasn't yet explored its testing capabilities. There is also a section on web testing that is independent of Rails.

I learned my lesson in Berlin

I still remember clearly the somewhat embarrassing mistake that sparked my interest in software testing. I was working as an intern for a research institute in Berlin and my task was to program an image processing algorithm in C. Once the program was finished I was stalled for a long time, maybe weeks, puzzled over the unexpected outputs of my program. My supervisor was starting to lose patience with my lack of progress. Eventually it occurred to me that the underlying algorithm that I had implemented was broken due to a trivial mistake. All along I had simply assumed that it was working without conducting any thorough tests to prove my assumption. That's when it dawned on me what a challenging and important problem software testing is.

The miserable past of web testing

A lot of developers see testing as a dull or even unnecessary activity. The level of testing has been particularly poor in web development, a sector of the software industry notorious for being hackish and low in quality. Granted, the cost of a broken web page is typically not as big as a bug in banking software. However, a broken page on a high volume public site or an internet banking site can be very costly indeed.

With Agile methodologies such as Extreme Programming the idea is to make changes, refactor, and release often. Good automated test coverage is almost a prerequisite for this style of development. This is because for each change that we make we need to make sure that we didn't break existing functionality. In a system with high coupling the bug caused by a change can arise in a seemingly unrelated part of the system. Of course, manual testing is not really an option in agile development as it would be too expensive and slow down the development and release cycle too much.

There are benefits to testing other than just finding bugs that are worth mentioning. One is that testing forces you to focus on and define the interface (aka contract) of your application or API. This is the reason why test driven development can be so valuable since defining the contract of your code early on provides a good direction and goal for your programming - it serves as a specification. Also, when you are writing tests you will often find yourself reviewing your code and this review in and of itself can reveal valuable improvements.

Rails shows the way

Most of the hype surrounding Rails has to do with its productivity and the surprisingly few lines of code needed to build a typical website. A nice side effect of having a small code base is that you have fewer lines of code to debug. Testing in Rails is also helped by its strict MVC (Model-View-Controller) structure. Last but not least, Rails comes shipped with a tightly integrated testing framework that allows testing both on the controller and model levels.

Rails has an interesting code statistics tool invoked by "rake stats" that counts the lines of code (LOC) and the ratio of production to test code. Here is the output for my website:

+----------------------+-------+-------+---------+---------+-----+-------+
| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Helpers              |    19 |    18 |       0 |       1 |   0 |    16 |
| Controllers          |   260 |   210 |       6 |      37 |   6 |     3 |
| Components           |     0 |     0 |       0 |       0 |   0 |     0 |
|   Functional tests   |   340 |   253 |      10 |      45 |   4 |     3 |
| Models               |   122 |    91 |       4 |      12 |   3 |     5 |
|   Unit tests         |   191 |   150 |       3 |      17 |   5 |     6 |
| Libraries            |     0 |     0 |       0 |       0 |   0 |     0 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total                |   932 |   722 |      23 |     112 |   4 |     4 |
+----------------------+-------+-------+---------+---------+-----+-------+
  Code LOC: 319     Test LOC: 403     Code to Test Ratio: 1:1.3

Unit testing

A unit test in Rails tests a single model class. For my website I have three model classes - Post, Category, and User. There is a one-to-one mapping in Rails between model classes and database tables. The model classes have methods for the usual CRUD (create, read, update, delete) operations and all of these operations interact with the database.

When you write tests that depend on database data it will simplify your life a lot if the tests have control over that data. This is one of the areas where the Rails testing framework shines. All Rails tests are executed against a separate test database into which data is loaded with the help of fixtures. Each database table has a corresponding fixture YAML file. YAML is a concise and yet readable text data format that is quite handy to work with. The YAML files can contain embedded ruby code and so can be dynamic with loops and variables.

Here are some examples of what my unit tests cover:

Functional testing

In Rails functional tests are applied at the controller level. Each controller class has a corresponding controller test class. The controller test executes actions on its controller, makes assertions about the values of the member variables in the controller, about the type of response (success, redirect etc.), and about the HTML output generated by the view. The controller test also has access to the request and response objects, to the session, and to the flash object used to display status messages to the user. All in all this makes controller tests very powerful.

Here are some functional tests I wrote for my website:

One minor limitation of controller tests to be aware of is that they don't include the URL-action mappings (the routes). One could probably test those separately somehow but I haven't seen an example of that yet. The URL mappings are covered by the web tests though, see below.

There is potential for overlap between unit and functional testing. Thorough testing at the model level decreases (but does not eliminate) the need for testing at the controller level. It is clearly an advantage if the controller tests can rely on the underlying model to be working correctly so that they can focus on the controller specific logic.

Web testing and Monitoring

I use web testing for pre-launch and post-launch acceptance testing and for monitoring of my website. This means I run the web tests just before a site upgrade, immediately after the upgrade, and then regularly every day against the production site. The web tests execute the entire system including the production database. The fact that the production database is used is important since it is not uncommon for bugs to depend on database data. The Rails unit and functional tests can provide good test coverage but the web tests are a necessary complement and are the ultimate check that the system is working at the point of release and that it keeps working in production.

I have divided my web tests into two parts. I have one "ping" test that I run frequently and that just makes sure that the front page works and the site is up. The other test is a smoke test that I run twice a day. For all pages on my site it checks that:

If any of the web tests fail I am alerted by email. I am also alerted by email if an error occurs in the code (i.e. an exception is thrown). With this setup I can rest assured that if major functionality on the site breaks down I will know about it and have a good chance of taking action quickly.

Web testing is not provided by the Rails framework and I have chosen to rely on Perl scripts for those tests. My web tests currently don't do much HTML parsing, but if there was a need for that Perl has its simple yet powerful look_down method. However, if there were similar APIs available in Ruby I would probably switch to Ruby as I am more comfortable with that language.

What's Missing?

There should be some kind of automatic test that verifies that site backups are working. A way to do that would be to have a mirror site in a different location that is updated nightly with the database backup from the production site. One could then run the web tests against the mirror site as well. One would also need to verify, through a web test or otherwise, that no data is missing in the backup. If the production site is not changed during the recreation of the mirror site a sufficient test would be to check if the two sites are identical.

If a website makes heavy use of JavaScript and Ajax then an in-browser testing tool such as Selenium may be useful for acceptance testing and browser compatibility testing.

Last, but not least, if you are expecting a lot of traffic you should look at load testing with tools such as Apache Bench (ab), or siege. Basic usage of these tools is shown in the Rails book. Rails also has the concept of performance tests that make it easy to load huge quantities of data and then repeatedly execute some critical action and measure its average execution time.

Conclusion

The level of testing that I have described here isn't comprehensive and really ought to serve as a baseline for pretty much any serious website. Unfortunately, this is far from the case in the industry today. With the advent of Rails, testing is more readily available to web developers than ever before. Let's remind ourselves to take full advantage of that.

1 Comment

Tue December 06, 2005
Programming

The Threat of New Technologies

In the midst of one of the many Java/Rails flamewars (haven't we had enough of them already?) I found a thought provoking comment by the creator of Tapestry on Rails and the resistance to technology change among developers:

"It's natural to be a little scared of new developments. It's painful to think that your exclusive, hard-won skills may be even a little bit obsolete. When I see Flex, Laszlo or Rails, my stomach does a little flip-flop in terms of all the effort I've put into Tapestry. But the reality is that *all* of these approaches are transitory (I'm sure we'll all be doing something quite different in five years), and we should embrace change, learn from other's efforts, and create even better frameworks and applications."

"For example, there may come a time when much Tapestry work is done using Trails (a Tapestry/Rails/Spring/Hibernate hybrid). Trails may never have offer quite the same developer productivity as Rails, but it is still a gateway into full-blown Tapestry and J2EE development."

Openness to new programming languages and toolkits seems to be a common characteristic among leaders in the programming community. The best example of this is probably the Pragmatic Programmers who came up with the now famous advice to learn one new programming language a year. I can't help but wonder that maybe if Dave Thomas hadn't picked up and documented Ruby, David Heinemeier Hansen would not have discovered it either and Rails would not have been a reality today.

I have been impressed that so many Java heavy weights (David Geary/JSF, James Duncan Davidson/Ant, Bruce Tate etc.) have had the courage to step up and recommend Rails. Personally I'm certainly ready and well equipped to make the transition to Rails and the urge to do so is bigger than ever now that I am back working in the less productive and overly complicated Java world.

Make a Comment

Sun November 27, 2005
Programming

I'm on Rails Now

I was finally able to put my new homepage live late last night after too many hours of programming (why do things always take longer than we expect?). About two weeks ago I caught a cold and I spent several days alone in my apartment. The upside of being sick is that I've had time to work very intensely on my homepage (when I probably should have been resting)...

This is the third incarnation of my homepage. A couple of years ago I started out with an OpenACS site using Lars Pind's weblog software. Since OpenACS hosting is hard to come by and that technology had little momentum I moved to dreamhost.com and Wordpress which both have worked out very, very nicely for me. At that point I didn't know the Rails revolution was coming...

When I was working for Collaboraid with Lars last year we had David Heinemeier Hansson (the creator of Rails) come visit us in our office and tell us about his toolkit. I remember how we were shooting all kinds of tough questions at him of the type "ok, that sounds good, but can Rails do this or that particular thing that we need". David exuded confidence and would patiently and immediately answer each and every one of our questions like "sure you can do that, or in fact, you can do something even better, let me show you...". We were all very convinced. At that time though, I didn't have the time or energy to pick up a new web technology and I was somewhat deterred by the fact that Rails was still such a small niche. Having lived in the OpenACS niche for so long I was very impatient to find my way back into the mainstream.

Rails stayed in the back of my mind though and when I came to Stockholm to work as a Java developer at Pricerunner I noticed how people were talking about Rails. I then realized just how big the technology must have gotten. Over the past months I've read countless articles about Rails and it's really mind boggling to see how Rails had litterally taken the world by storm. I didn't see that coming, at least not at the rate that it's been happening. There is a myriad of open source web toolkits out there competing for attention and people like me who have worked with OpenACS know how hard it is to get a toolkit using an unpopular programming language into the mainstream (there are other more compelling reasons why OpenACS never entered the mainstream, but that's a different story).

When I had realized that the Rails way is really the future of web development, and also had experienced first hand how wonderful the Ruby language is, I knew I had to learn these technologies. I figured that what better way to do that than to build my own homepage using them? I started by trying Typo - the Rails weblogging software that all the other cool Rails hackers seem to be using. However, at the time that I tested it I was kind of put off by some bugs that it had. It seemed awfully immature coming from Wordpress. Also, Rails is not a framework for plugging in and reusing applications (like OpenACS is). Rather, it is a framework for building applications from scratch in a very fast and elegant fashion.

Of course, building any application, even if it's something seemingly simple as a weblog, is going to take a lot of work and thought if you want the result to be top class. What's fascinating is that Rails makes it feasible (because of its productivity) to build from scratch in a lot of cases where previously application reuse was the only realistic option.

I'm really thrilled about web development again know that I know Rails. I'll write up some details about how I built my homepage later on.

My old homepage is still accessible and now lives at old.marklunds.com.

1 Comment

Fri November 25, 2005
Programming

Java is not Always the Best Tool? Really?

I really like the article Ruby the Rival over at OnLamp.com, and especially this part by James Duncan Davidson (this has been said many times before - when all you have is a hammer everything looks like a nail):

"And I think that's the real win from the recent attention on Ruby on Rails and the breakaway from viewing the world with Java-colored glasses. It's not that Ruby on Rails is going to be the next Java. Far from it. It's that Ruby on Rails helps to break this idea that there is "One True Way." There's not. There are many different ways to solve a problem. And really, none of them is the clear-cut winner. There's just places where one solution has advantages."

Make a Comment

Fri September 23, 2005
Programming

David Heinemeier on Rails Scalability and the Importance of Programmer Culture

A .Net programmer asked on the Rails mailing list if the switch to Rails had really been worth it for him and whether Rails scales. David Heinemeier gives a nice and balanced reply pointing to the scalability of BaseCamphq.com and 43things.com but also stressing the importance of being in a programmer culture that you like.

By the way, I am very impressed by the Gmane mailing list web site that provides a very usable threaded web interface to mailing lists that is searchable. Kudos to the Gmane people (apparently in Norway) for volunteering to offer such an excellent service to the programming community! I feel like donating some money to them.

Make a Comment

Sun March 27, 2005
Programming

Voila - My new Homepage

As a result of my move to Stockholm and no longer working for Collaboraid I have now moved my homepage over to a new server at Dreamhost - an excellent LAMP hosting company in the US.

For my weblog I chose to use Wordpress and I have certainly not been disappointed. In fact, Worpress continues to impress me in so many different ways. The UI is very slick and the software feature rich. It has a great community with many themes and plugins, and it has seemed robust so far.

Most importantly, I was totally blown away by the simplicity and elegance with which I could extend Wordpress using a simple callbacks / hooks mechanism. I was able to quickly add a language filter for posts allowing users to view only posts in english, and I managed to do this without modifying the Wordpress source code!

I've only looked at the Wordpress codebase briefly, but it seems to me they are following conventions whenever they can, and keeping things simple. It's the beauty of the PHP software I've seen so far - it's very easy to find your way around the software and make small hacks and get immediate feedback on your changes.

Wordpress retains most of the typical PHP simplicity and straightforwardness and combines it with good design, very little duplication, and most notably a brilliant callback extension mechanism that preserves upgradability. I hope this kind of callback approach is more widely adopted in the industry. I think it is the way forward and finally I am hopeful that we can solve the annoying problem that has been with us for so long of allowing open source customizations and upgrades to play well together.

Make a Comment

Thu September 23, 2004
Programming

A Skeptical Perspective on the Open Source CMS Market

I am helping Greenpeace build a new Open Source CMS system based on OpenACS and I stumbled across the article Open-Source CMS: Prohibitively Fractured? by Tony Byrne, founder of CMSWatch. I think the article makes some good points for example about programming language religion, technical elegance over end user features, dilusion of efforts, the risk of forking, and re-invention of the wheel over reuse of standard APIs (for example from the Apache project).

There is quite an impressive supply of sites on the web providing listings, reviews, demos, and articles about CMS systems, such as OpenSourceCMS.com, CMSMatrix, and CMSInfo

Make a Comment

Thu August 26, 2004
Programming

Floating in Hosting Paradise

I just signed up with DreamHost and I must say that so far it's been an amazingly smooth and pleasant experience. Their web control panel and help systems are really powerful and usable and everything on the server has worked great so far! DreamHost really seems to have great sys admins and they've been in the business for quite a while so they've probably learned a lesson or two over the years.

I really like managed hosting, it gives you that warm and cosy feeling of beeing well taken care of and not have to worry about things like backups and server uptime and other boring infrastructure details. Of course you still need to think about backups but at least your hosting provider has a fallback for you if your own setup fails.

Here is a list of features at DreamHost that I copied from one of their informative and amusing newsletters:

* Jabber Server.
* Quicktime Streaming.
* Miva Merchant included on a pretty cheap plan.
* We keep two hourly, two daily, and two weekly FULL backups of all 
   websites on all plans for free, and you can access them in the hidden
   .snapshot directory in every directory of every user.
* All Debian.
* Lots of Domains/SubDomains/Users/Databases/Disk/Bandwidth.
* DreamHost Rewards.
* DreamHost Donations.
* We're small(ish), never merged with anybody, and not public.
* The Sexiest Newsletter Writer in the Industry, nay, the World.

Make a Comment

Tue June 01, 2004
Programming

Software Project Methodology

I am reading "Software Project Survival Guide" by Steve McConnell which I find very useful. Since McConnell is talking about upstream and downstream activities (waterfall approach?) and emphasizing that changes are vastly costlier when they are made late on in the project, I was curious as to how compatible his approach is with Extreme Programming. The following links provided some insight:

I found this section from McConnell's speech particularly illuminating:

Design Advice -- What has changed in 10 years?

    * In 1990s, design pundits wanted to dot every i and cross every t before writing any code
    * In 2000s, say BDUF (big design up front)? YAGNI (you aren't gonna need it)
    * There are lots of valid points on the no design--all design continuum

Extremes are usually not productive

    * All design up front vs. no
    * Entirely planned vs. improvised
    * Pure iterative vs. straight sequential
    * All structure vs. all creative
    * Document everything vs. nothing

Maybe it is in the overlap of the different software project management approaches that we, using our common sense and experience, can find the method that works best for the particulars of our own software projects.

In the book "Extreme Programming Perspectives", Pascal van Cauwenberghe offers this acronym for finding the balance between up-front work and refactoring:

"Instead of using the disparaging term "big design up front" (BDUF) we should be investigating how best to determine what is "just enough design for increments" (JEDI)."

Make a Comment

Sat May 29, 2004
Programming

Opening a File

Bruce Eckel on why he loves Python:

They say you can hold seven plus or minus two pieces of information in your mind. I can't remember how to open files in Java. I've written chapters on it. I've done it a bunch of times, but it's too many steps. And when I actually analyze it, I realize these are just silly design decisions that they made. Even if they insisted on using the Decorator pattern in java.io, they should have had a convenience constructor for opening files simply. Because we open files all the time, but nobody can remember how. It is too much information to hold in your mind.

Here is how you open a file in Python: "for line in file('filename').readlines():,". I'm increasingly motivated to learn Python and also to have a closer look at Plone.

Make a Comment

Sat April 17, 2004
Programming

PHP - Ugly but Simple

It bodes well for the future of PHP when Rasmus Lerdorf writes that:

Despite what the future may hold for PHP, one thing will remain constant. We will continue to fight the complexity to which so many people seem to be addicted. The most complex solution is rarely the right one. Our single-minded direct approach to solving the Web problem is what has set PHP apart from the start, and while other solutions around us seem to get bigger and more complex, we are striving to simplify and streamline PHP and its approach to solving the Web problem.

4 Comments

Sat October 11, 2003
Programming

Joel Spolsky on Internationalized Software Development

I always find Joel Spolsky's articles entertaining and witty. His recent one on encodings and character sets I found especially useful as I'm acticely involved in the internationalization of the OpenACS web platform. The internationalized and better than ever 5.0 version of OpenACS is scheduled to be released on November 1st and I have some hectic weeks leading up to the release ahead of me now...

Make a Comment