Peter Marklund

Peter Marklund's Home

Peter on Software

Lessons Learned in software and web development Subscribe

2007-05-01T05:13:45.000Z

Bug Tracking can be Bug Prevention

I've been deploying a couple of Rails applications for a client lately and things have been running surprisingly smoothly and there have hardly been any issues at all. In one of the more complex applications though a few bugs slipped through my testing net and were caught by the excellent Exception Notifier plugin by Jamis Buck. Now, the question is, how do we typically deal with bugs as programmers? If the bug is in production we rush to fix it, push the fix out, and hope not too many people noticed it, and then try to forget about the whole thing. What we should be doing though is ask ourselves why the bug occured in the first place, how we can categorize the bug, and what kind of process and/or tests can we put in place to prevent similar kind of bugs from happening in the future. Bug tracking is a gold mine for figuring out how we can raise the quality of our software and reduce the bug rate in the future. This may be common sense, but how many of us actually use this opportunity?

2007-03-28T08:45:01.000Z

Rails Gotcha: No ordering of ActiveRecord after_save callbacks

I am setting up tagging and searching for a demo application and I am combining the plugins acts_as_taggable_on_steroids and acts_as_ferret. I was using the following setup for my users:

  acts_as_ferret :fields => [:login, :email, :bio, :tags_string]

  def tags_string
    tags.map(&:name).join(" ")
  end

However, that didn't work. When making a change to my tags I had to save the user twice for the search index to be updated. Why? Because the acts_as_taggable_on_steroid after_save callback (save_tags) runs after the acts_as_ferret after_save callback (ferret_update). Maybe the callbacks are invoked alphabetically, I don't know. Anyway, there is no explicit ordering and no after_after_save callback...

The workaround in this case was easy because of the virtual attribute tag_list that gets set before the save:

    def tags_string
      Tag.parse(tag_list).join(" ")
    end

Now we are always using the latest tag list when we update the Ferret index. Problem solved.

2007-03-26T11:21:09.000Z

Rails Extension: ActiveRecord::Base.find(:last)

I'm at the Big Nerd Ranch Ruby on Rails bootcamp in a monastery outside Frankfurt right now having a great time. One of the students asked if you can say ActiveRecord::Base.find(:last), which you can't. The value of the first argument is typically :first, but you can't use :last. Just for educational purposes I decided to change that (it's probably not something I would use in production):

ActiveRecord::Base # Make sure class is loaded, which it probably is by now anyway
module ActiveRecord
  class Base
    def self.find_with_last(*args)
      if args.first == :last
        options = extract_options_from_args!(args)
        find_without_last(:first, options.merge(:order => "#{primary_key} DESC"))
      else
        find_without_last(*args)
      end
    end

    class << self # Needed because we are redefining a class method
      alias_method_chain :find, :last
    end    
  end
end

I guess most interestingly this code illustrates how to alias a static method.

2007-03-26T03:20:06.000Z

Abandoning Textilize for Comment Formating

I've been using the textilize helper for comment formating in this blog. The textilize helper uses the RedCloth gem, which supports both Textile and Markdown syntax and it's pretty complex code with lots of hairy regular expressions. I noticed that the textilize helper will let arbitrary HTML tags through so it can mess up the HTML of a page. Seems like a vulnerability, and it's even more a problem for me since I like to be able to HTML validate my pages. I decided to stop using textilize for comments and instead use the following simpler and tighter formating:

  # HTML quote, convert line breaks to <br />, and convert URLs to links
  def simple_link_format(text)
    # From http://snippets.dzone.com/posts/show/3654
    url_regex = /(http|https):\/\/[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,5}(([0-9]{1,5})?\/.*)?/ix
    simple_format(h(text).gsub(url_regex, '<a href="\0">\0</a>'))
  end
2007-03-26T02:04:32.000Z

Capistrano is not just for deployment

I'm appreciating Capistrano more and more. It has such an elegant and powerful API, and you can use it for pretty much any commands you need to execute on one or more of your servers. Here is a simple example. For the development of this weblog I found myself quite often wanting to copy the production data to my development server. Of course at first I was doing this on the command line. The next natural step would have been to write a bash script, but these days I tend to write most of my scripts in Ruby. Rather than stick a small Ruby script under the Rails script directory I'm starting to create Rake tasks under lib/tasks though. One advantage of having scripts as Rake tasks is that you can cagalog them nicely with "rake --tasks". Here is the Rake task:

namespace :db do
  desc "Update the development database with a fresh dump of production"
  task :update_dev do
    host = "deploy@marklunds.com"
    db = "marklunds_production"
    system("ssh #{host} \"mysqldump -u root --set-charset #{db} > /tmp/#{db}.dmp\"")
    system("scp #{host}:/tmp/#{db}.dmp ~/tmp")
    system("mysql -u root marklunds_development > ~/tmp/#{db}.dmp")
  end
end

Looking at this code I realized it needed information in the Capistrano config/deploy.rb file, such as the domain of the production server. Also, what the task really does it execute some commands on a remote server, and that's exactly what Capistrano is built for. So I moved the task over to deploy.rb:

require File.dirname(__FILE__) + '/../config/environment'

...the usual deploy.rb stuff here...

desc "Copy production database to development database"
task :update_dev_db, :roles => :db do
  prod_db = ::ActiveRecord::Base.configurations['production']['database']
  dev_db = ::ActiveRecord::Base.configurations['development']['database']
  run <<-CMD
    mysqldump -u root --set-charset #{prod_db} > /tmp/#{prod_db}.dmp
  CMD
  system("scp #{user}@#{domain}:/tmp/#{prod_db}.dmp ~/tmp")
  system("mysql -u root #{dev_db} < ~/tmp/#{prod_db}.dmp")
end

It seems a bit unclean that I have to source the whole Rails environment from Capistrano and it increases the startup time a bit. The reason I do that is to get the database names from ActiveRecord instead of hard coding them. Notice the flexibility of Capistrano tasks - executing local commands is just as easy as executing remote ones.

2007-03-25T14:33:42.000Z

Rails Deployment: Running Tests in Production with Capistrano

Murphy teaches us that if something can go wrong it will. Given all the differences there are between my development and production server - operating system, web server, database, gems etc. - I can't really feel comfortable deploying my application unless I have first run my test suite not only in development but also in production. Thanks to the beautiful and powerful API of Capistrano this is a walk in the park to accomplish:

desc "Run the full tests on the deployed app." 
task :run_tests do
 run "cd #{release_path} && RAILS_ENV=production rake && cat /dev/null > log/test.log" 
end

desc "Copy in server specific configuration files"
task :copy_shared do
  run <<-CMD
    cp #{shared_path}/system/voxway.rb #{release_path}/config &&
    cp #{shared_path}/system/database.yml #{release_path}/config
  CMD
end

desc "Run pre-symlink tasks" 
task :before_symlink, :roles => :web do
  copy_shared
  run_tests
end

In my current project I am deploying to two identical Linux servers and since Capistrano runs the tests in parallel on those servers I need to keep two distinct test databases in production - one for each server. Credit for this approach goes to Evan Henshaw-Plath who described it nicely.

2007-03-25T13:42:57.000Z

Rails Deployment: Check MySQL Charset on Startup

The UTF-8 support got a lot better with the Rails 1.2 release but since Rails has evolved so fast many of the instructions you'll find on the web are dated. Rails will now default the Content-Type of its responses to UTF-8 so you don't need to set that yourself in an after filter.

When it comes to configuring MySQL to use UTF-8 there are at least three pieces to keep track of - the database encoding, the client encoding, and the connection encoding. For my project it was enough to set "default-character-set=utf8" in the MySQL my.cnf config file (takes care of the database part) and set "encoding: utf8" in database.yml (takes care of the connection and client).

A really great idea though is to check that the proper encoding is configured when your server starts up in production on your various servers and abort if it isn't. Credit goes to Mike Clark and the Pragmatic Programmers for sharing the following useful code snippet:

if RAILS_ENV == "production"
%w(character_set_database character_set_client character_set_connection).each do |v|
  ActiveRecord::Base.connection.execute("SHOW VARIABLES LIKE '#{v}'").each do |f|
    unless f[1] == "utf8"
      puts "ERROR: MySQL database isn't properly encoded!"
      puts "Kindly set your #{f[0]} variable to 'utf8'."
      RAILS_DEFAULT_LOGGER.error("MySQL database isn't properly encoded!")
      exit 1
    end
  end
end
end

This is yet another example of failing early, one of my favorite design principles, and it seems to have become a running theme in this blog...

2007-03-24T16:35:50.000Z

Rails Testing: Making assert_select XML safe

I've been using the new assert_select command extensively in my tests lately and it's a wonderfully powerful tool - a huge improvement over assert_tag. The only annoyance has been the warning messages "ignoring attempt to close form with link" that have been polluting my test output. The other day I decided to track the source of the messages and I discovered that others had already complained about them. I found that the root cause of the problem lies in the HTML::Document and the HTML::Tag classes that ship with Rails. More specifically the problem is in the HTML::Tag#childless? method that auto-closes HTML tags such as img, br, and hr.

What do I mean by auto-closing. Well, in XML you have to close all tags, either with a trailing slash, or with a closing tag. In HTML however, we use tags such as <br> and <link>, usually without a trailing slash. The HTML::Tag class takes those special tags into account and understands that they are auto-closing (childless) without the trailing slash. This becomes a problem if you have an XML document that contains <link>some content</link> since the parser will think the first tag is self-closing and the second tag will be a mismatch. Not only does this cause warning or error messages, but it breaks assert_select for those specific tags.

I have submitted a patch that makes HTML::Document enter XML mode if the document has an <?xml declaration. Jamis has posted about his approach which he calls assert_xml_select. However, since all my documents are XML (XHTML or VoiceXML in my current project) I chose the following approach instead which allows me to use assert_select:

if RAILS_ENV == 'test'
  module ActionController #:nodoc:
    module TestProcess
      # Work around Rails ticket http://dev.rubyonrails.org/ticket/1937
      def html_document
        @html_document ||= HTML::Document.new(@response.body, true, true)
      end
    end
  end
end

Setting the first boolean argument to true makes the parser error out if there is an unclosed or mis-matched tag. I want my documents to be valid and I like to fail early.

2007-03-20T18:21:42.000Z

Rails Tip: Declare your Gem versions

Many Rails applications have external dependencies in the form of RubyGem libraries. The problem with such external dependencies is that you need to make sure that you have the right gem versions installed across all your servers, i.e. development, staging, and production. The probably easiest way around this is to freeze gems into your source tree. For rails you can use the Rake task "rails:freeze:gems" and you can freeze other gems by using Rick Olsen's Gems plugin.

At the end of the day there may still be external gems that you depend on, maybe because they need to be compiled, or maybe because you want to avoid your application source tree growing too big. You can fix the version of Rails using the RAILS_GEM_VERSION constant in environment.rb. For other gems you can fix the versions by adding statements like the following to your environment.rb:

gem "RedCloth", "3.0.4"

The gem command specifies that we will be using a gem of a certain version and it will attempt to add that gem to the load path (activate it), but not require it. If the gem version is missing you will be alerted on server startup.

So how do you know which gems you are depending on? One way might be to look at the load path. The following expression will return a list of Gems that have been activated so far in your application:

$:.map { |path| path[/^#{ENV['GEMS_PATH']}\/([^\/]+)/, 1] }.compact.uniq

You can inquire about a certain gem on your system using "gem list" and "gemwhich":

$ gem list|grep RedCloth
RedCloth (3.0.4, 3.0.3)
    RedCloth is a module for using Textile and Markdown in Ruby. Textile

$ gemwhich RedCloth
/usr/local/lib/ruby/gems/1.8/gems/RedCloth-3.0.4/lib/RedCloth.rb

Also, don't forget that if the gem comes with RDoc you can access that documentation by invoking gem_server on the command line.

2007-03-20T16:48:29.000Z

RubyGems: LoadError in nested require no longer causes silent failure with Ruby 1.8.5

I had a Rails application where I was trying to reference a helper that didn't exist. What happens in Rails when you try to use a class that you haven't required is that the Ruby callback method const_missing is invoked. Behind the scenes Rails will then try to require the class for you. What was happening was that Rails was trying to require the helper file, but the helper file didn't exist, and so that failed. The problem wasn't so much that it failed, but that it failed silently - there was no error message to be seen anywhere.

After having painstakingly tracked down the problem it turned out that it had to do with the Ruby require method and how RubyGems (I was using Rails through RubyGems) overrides the require method. RubyGems redefines the require method so that it first invokes the original Ruby require method, and if a LoadError is thrown, it will attempt to find the appropriate Gem, add it to the load path, and then invoke the Ruby require method again.

I posted about this silent failure on the RubyGems mailinglist and Eric Hodel suggested I should file a bug if I could reproduce the problem without Rails. Now several weeks later, and after an upgrade to Ruby 1.8.5, it turns out I can no longer reproduce the bug. The reason is that the Ruby require method behaves slightly differently with respect to LoadErrors in Ruby 1.8.5.

This may never be relevant to you, but let me just briefly try to explain the details of what is going on. Suppose you have a file called test.rb with a single line that says:

  require 'this_file_does_not_exist'

Then with Ruby 1.8.4 you would get:

$ irb
irb(main):001:0> require 'test'
LoadError: no such file to load -- this_file_does_not_exist
        from ./test.rb:1:in `require'
        from ./test.rb:1
        from (irb):1
irb(main):002:0> require 'test'
=> false

As you can see the second time you try to require the file the require method returns false, i.e. no LoadError is thrown. This indicates that the file has already been loaded. However it doesn't tell us that the loading failed. This absence of a second LoadError is what yields a silent failure if test.rb lives in an activated gem. The reason is that the first LoadError is caught by the RubyGems require method.

With Ruby 1.8.5 however a LoadError will be thrown everytime you do "require 'test'" and thus the LoadError will no longer be hidden by RubyGems. I don't know why they changed the contract of the Ruby require method, they might have had other reasons, but my suspicion is it was to avoid some kind of silent failures.

2007-03-02T04:17:21.000Z

Rails Gotcha: ActiveRecord::Base#update_attribute skips validation

It's something others have been bitten by. Although we should know better (why didn't we read the API docs carefully?) we naivley thought we could safely update a single attribute with the update_attribute method. When you invoke my_precious_object.update_attribute(:important_attribute, erroneous_value) then Rails will happily skip validations and save all attributes to the database. This is unlike the sister method update_attributes which updates several attributes and does use validation. I'm sure validation is skipped for a reason but I'm not sure what that reason is. In particular I'm not sure if that reason is strong enough to warrant the inconsistency and the risk of skipping validations.

I for one thought all public methods that create and update an ActiveRecord object were supposed to use validations. That would have seemed natural and even essential. Especially since most Rails apps run on MySQL which doesn't have a great track record when it comes to data integrity.

As far as I can see there is no mention in the Rails book of this validation glitch. I think I should pass a note to Dave Thomas about that.

If you look under the hood of Rails, it turns out update_attribute is equivalent to:

  object.attribute = value
  object.save(false)

The update_attribute method is first defined in the Base class and then overridden by the mixed in Validations module. To see the definitions of the save, update_attribute, and update_attributes methods you would be looking in vain in the Base class, in fact, the code you see there is never used, which feels a little bizarre. Fortunately there are comments in there pointing us to the Validations module.

On a side note, notice that unlike in Java, there is no method overloading on argument signatures in Ruby, so a method is identified solely by its name. When you redefine a method you can effectively change its argument list if you like, although this is probably not a good idea in most cases. The Validations module uses this ability though for the save method to add an optional boolean argument that determines if validation should be performed or not. This in turn is what the redefined update_attribute method makes use of. Questions? :-)

2007-03-02T03:18:12.000Z

Nokia N70 - 11 Clicks Away From Usability

I've had the privilege and misfortune of using a Nokia N70 3g camera phone over the last couple of days. My first impression of the phone has been mostly a series of frustrations but most importantly a lesson in the importance of usability and delivering on the basics. To quote 37Signals:

The basics are the secrets of business. Execute on the basics beautifully and you’ll have a lot of customers knocking at your door. Cool wears off, usefulness never does.

Well, if 37Signals had consulted for the Nokia N70 development team I'm sure its UI and featureset would have been radically different.

To use a phone I need to know how to turn it on and off, how to disable the keyboard to put the phone in my pocket, how to put the phone in silent mode, how to make normal old fashioned phone calls, how to store friends phone numbers, and how to send SMS (including how to turn the auto completion on and off and write special chars like punctuation). That's about it. That's the full list of essential features that I need to master in order to pick up a phone and get by with it in my everyday life.

There is too much noise in the N70 UI, the essential features are not prominent and accessible enough. All else being equal, the more features you add, the more noise, the less obvious the basics will be, and the lower the usability. The user manual mirrors the N70 UI in being unstructured and not highlighting the basics. For example, I would have expected the first chapter to tell me what's in the package, have a picture of the phone and explain essential buttons and functions, how to turn the phone on and off etc. The Nokia N70 manual doesn't care for this approach but instead dives straight into advanced features such as how to configure the camera.

Another reason for low usability is not following conventions. The N70 phone is in between a Windows desktop UI and the UI of a second generation phone (remember the phones that didn't have cameras and internet?). It has its own set of conventions and before you have loaded those into your brain you are likely to be left confused and frustrated. Clicking the "select" button placed to the left on the phone is the equivalent of the right mouse button in Windows. Ok, so left and right are confused, we can live with that I suppose. Well, hold on, that analogy doesn't even hold because some of the options you get when you press "select" are not related to that particular item but to the whole list of items...

Putting the phone in silent mode is something I need to do often and its about 11 clicks on the Nokia N70. At the same time the desktop (start screen) is mostly empty and as far as I can tell not configurable, and several buttons on the start page are left unused.

Let's leave this ramble on a positive note - taking pictures with the phone is easy! Just slide open the camera and press the button. I figured that out on the first try. Except, the annoying thing about the camera is that it slides open by mistake and is left in camera mode in my pocket. This might explain why the battery drains so quickly, or maybe that is because of the large screen and all the software. Oh well, I try to be positive, I really do...

2007-01-31T03:34:29.000Z

Swedish Rails Article and Course

I was very pleased to see my Ruby on Rails article published yesterday by the IT education company Informator here in Stockholm. The article is in swedish and has a title along the lines of "Rails Brings Back the Joy of Working". It was published in the slick offline paper called "Format" that Informator distributes and is also available online.

I there is enough interest (fingers crossed) I will be giving the Ruby on Rails course for Informator here in Stockholm later this year. Needless to say, I'm very excited about that opportunity!

2007-01-06T04:02:27.000Z

Rails Tip: Use the Unit Tests as Documentation

If you are looking for authoritative answers on how to use the Rails API, look no further than the Rails unit tests. Those tests provide excellent documentation on what's supported by the API and what isn't, how the API is intended to be used, the kind of use cases and domain specific problems that drove the API, and also what API usage is most likely to work in future versions of Rails.

Let me give you a concrete example. In my code I had the equivalent of a Tag class connected through a Tagging join model with two different classes, say Post and Comment. What I was trying to do was say in the Tag class something like "has_many :posts, :through => :taggings, :source => :taggable" or maybe "has_many :taggables, :through => :taggings". However, Rails refused to let me do this and kept throwing an ActiveRecord::HasManyThroughAssociationPolymorphicError. At this point I wasn't sure if I was misusing the API or if I had come across a Rails bug (although I knew that the former was more likely).

I turned first to the most authoritative documentation such as the API doc, the Agile Web Development with Rails book, and the Rails Recipes book (Recipe 22: "May-to-Many Relationships with Extra Data", and Recipe 23: "Polymorphic Associations - has_many :whatevers"). However, none of that documentation seemed to address my specific problem. Through Google I found the wiki page HowToUsePolymorphicAssociations which did address my problem. However, the solution there seemed clumsy, and I didn't know how authoritative and up-to-date it was.

It's usually a good idea to debug a problem by following the stack trace into the Rails code and try to figure out what's going on right there. However, in this particular case I found that reading the clean but complex ActiveRecord code didn't give me any easy answers. It then hit me that what I really should be doing was read the ActiveRecord unit tests. Luckily for us application developers David and the rest of the core team has provided us with a nice suite of unit tests. Once I looked at the join model test case the whole picture cleared up - the fog lifted and there was bright sunlight. Laid out right there in front of me were excellent and authoritative examples of how to use ActiveRecord associations with classes such as Tag, Tagging, Post, Comment, Category, and Categorization. I also found the particular assertion that said that the ActiveRecord::HasManyThroughAssociationPolymorphicError should be raised in my case...

One could speculate that Rails really should provide a way to do has_many through a polymorphic association, and surely there is a way to hack Rails into doing something like that. What's important though is that now I know that the approach is not officially supported or intended and that's fine with me. I like to stay on the unit tested golden Rails path and I recommend you to do the same.

2007-01-04T10:29:50.000Z

Good and Bad Programmer Practices

As programmers we have a tendency to take shortcuts that come back and bite us eventually. It is very popular to write overly complex system that seem sophisticated and make us feel smart and special. The irony is that a lot of times we think our code is so simple and obvious that it doesn't need automated tests. Here is a list of programming practices to consider:

In summary then, keep your code clean, simple, conventional, and well tested and listen to your users. Those recommendations may sound obvious, in practice though they often contradict our instincts as programmers and require discipline and courage to stick to.

2006-12-27T10:23:56.000Z

New Rails Plugin for HTML Validation: Html Test

Update on 2009-04-16: The source code has been moved from Google Code to GitHub

Update on 2007-12-12: I have renamed the plugin from "Http Test" to "Html Test" and moved it to a new home at Google Code. Sorry about the inconvenience that this may cause those of you who have the plugin installed and want to update it. The change is essentially a namespace change from Http:: to Html:: so it should be manageable. Thanks.

As I wrote earlier I like to monitor my production sites with HTTP tests that validate their HTML and check for broken links and images. I think that serves as a good baseline check, a smoke test if you like, that the site is not fundamentally broken.

It used to be that my HTTP tests were implemented in Perl using WWW::Mechanize. Now that we have the wonderful Ruby and Ruby on Rails at our fingertips that felt like a real shame. I decided to implement my HTTP tests in Ruby and once I had gotten that far, going the extra mile to package my scripts as a Rails plugin seemed natural. The plugin is called http_test and it's available at Google Code.

So far, what the plugin does is mostly provide support for HTML validation, and to some extent link checking. It provides a command line script that you can use to conveniently HTML validate your site, either in production, staging, or development, as you please. You give the script a URL and it will fetch and validate the page over HTTP. It will also follow links and validate those pages. The script is not a web spider though. It only follows links one step away from the start page.

The other thing you can use the plugin for is to validate pages in your controller and integration tests. The plugin provides the assertion trio assert_tidy, assert_w3c, and assert_xmllint. There is also the catch all assertion assert_validates that can invoke all three validators for you.

What I like to do is setup my application controller to validate all my requests in the test environment. That way you know that all pages that you request in your functional and integration tests validate. The plugin makes this easy. All you have to do is add the following lines to your test_helper.rb:

    ActionController::Base.validate_all = true
    ActionController::Base.validators = [:tidy]

To be able to use Tidy the http_test plugin relies on code in the rails_tidy plugin, so credit to Damien Merenne for providing that. Credit also goes to Scott Baron for the W3C validation, and to Maik Schmidt, author of "Enterprise Integration with Ruby", for the xmllint validation code.

I hope the Html Test plugin will be useful to others in their HTML validation and link checking pursuits. Please let me know what works and what doesn't and how I can improve. Thanks!

2006-12-12T07:37:00.000Z

Hype and the Rise and Fall of Programming Languages

Steve Yegge has written an article on Ruby as a competitor to Python, the Rails hype, and the rise and fall of programming languages. The article touched me deeply. Thanks to Olle Jonsson for sending the link. The article is almost a year old but should still be relevant. I especially like the last section on Ruby:

"The worldwide Ruby culture is the warmest and friendliest I've seen in my long history with programming languages. And Ruby is a sweet language. Other people seem to agree, and are taking steps to market it, which is getting them labeled as "hyper-enthusiasts" by the Sour Grapes camp. It appears to me that Ruby is doing what I wanted Python to do a few years ago, so I've finally learned Ruby and have switched most of my development over to it.

After all, both languages have a long way to go before they catch up with Java in terms of tools, IDEs, books, performance, threading stability, and tons of other stuff. I wanted to make a reasonably educated bet, and choose the language I think is going to be bigger, so it'll work well for me, and so I won't have to fight so hard to use it in my job.

It wasn't hard to learn Ruby. In fact after a few days with it, Ruby felt as comfortable as languages I'd been using for years. I've really been enjoying it. It has a few warts; all languages do. No biggie. It looks as if Matz is intent on fixing them in Rite.

I don't know if I like it more than Python and Scheme. I like it at least as much as those languages, certainly. But Ruby's my favorite (as in "preferred") language now because I can see the trajectory it's on, especially now with Rails, and I believe it's going to be the Next Big Thing -- a Java-like phenomenon. So did Bruce Tate when he wrote "Beyond Java". So do James Duncan Davidson, Dave Thomas, Martin Fowler, and many other people who are a heck of a lot smarter than me. You'd think they're on to something big, wouldn't you? I do.

Java-like worldwide adoption really matters. Without that level of mass-market adoption, Ruby won't get the tools, stability, and CPAN-like library selection that it needs in order to compete with Java and Perl. It's a chicken-and-egg problem that all languages face, and Ruby stands a chance of succeeding where Smalltalk, Python, and other great languages have (to date) failed.

I see Rubyists worrying that Rails is stealing the show. Geez, folks, LET it steal the show. Talk about a free ticket for Ruby success. Java Applets were a way to get Java in front of a million or so programmers, ultimately allowing the Java platform to succeed in all sorts of domains that it might never have seen without the initial "killer app" of Applets.

We live in a world where culture matters, economics matter, and marketing hype matters. They are very real forces that directly affect our quality of life as programmers. You ignore them at your peril, a lesson learned by so many almost-forgotten languages that were stomped by marketing hurricanes like Java and Perl.

I really wanted Python to succeed, and I still wish them the best, but I think they're ignoring marketing. I really want Ruby to succeed, so I get a bit miffed when I hear famous people like Bruce Eckel making uninformed generalizations about both Ruby and the folks who are working hard to make it successful. I think Pythonistas should be focusing on doing the same -- working to make Python successful. I do think it will take a minor cultural adjustment on their part. And they need to start accepting hype as a natural part of the world we live in, a requirement for cutting through the noise. But I think they can do it. "

2006-12-08T05:33:00.000Z

MySQL Gotchas and a Deadly Sin

Coming from mostly a PostgreSQL and Oracle background I experienced some major friction working with MySQL today.

The first obstacle to overcome was the syntax for foreign keys. I started out using

  create table test (
    service_id integer NOT NULL references services(id) on delete cascade
  );

MySQL 5.0.21 on OSX swallowed this syntax happily with no complaints. Only by running Rails unit tests did I discover that to actually create a foreign key in MySQL (and not just look like you are) you need to write:

  create table test (
    service_id integer NOT NULL,
    foreign key (service_id) references services(id) on delete cascade
  );

When I then actually had working foreign keys the ordering of table create and drop statements became very sensitive. If table A has a foreign key to table B then table B must be created before table A. Also, MySQL lacks the DROP TABLE CASCADE feature so the order of drop table statements needs to be the reverse of the create table statements. In this case (apparently for porting reasons) you can still write DROP TABLE CASCADE but MySQL will just ignore the CASCADE command. This, just like the foreign key example above, is a form of silent failure that I think in general is the root of much frustration among developers. It's an anti-pattern.

Now here comes the grand finale and my worst discover when it comes to MySQL today - the implementation of NOT NULL. You see in this case, the NOT NULL instruction is not ignored by MySQL, but arguably it's not fully implemented either (see a good discussion on this here). It turns out that if you try to insert NULL into a NOT NULL column MySQL throws an exception. This is to be expected and is exactly what we want. However, if you instead try to update the column to NULL MySQL happily sets the column to the empty string...

CREATE TABLE peter (
  name varchar(100) NOT NULL
) ENGINE=InnoDB;

mysql> insert into peter (name) values (NULL);
ERROR 1048 (23000): Column 'name' cannot be null

mysql> insert into peter (name) values ('foobar');
Query OK, 1 row affected (0.06 sec)

mysql> update peter set name = NULL;
Query OK, 1 row affected, 1 warning (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 1

mysql> select count(*) from peter where name = '';
+----------+
| count(*) |
+----------+
|        1 | 
+----------+
1 row in set (0.00 sec)

mysql> select count(*) from peter where name is null;
+----------+
| count(*) |
+----------+
|        0 | 
+----------+
1 row in set (0.00 sec)

I'm at a loss of words...

2006-12-03T03:50:00.000Z

Usability on the Web and Embracing Simplicity

I just booked flight tickets for the christmas holiday and it got me thinking of usability. People depend on websites so much these days and it's a shame to let them suffer from poor usability. Of course, usability has improved quite a bit over the last couple of years. Indeed SEB finally uses a date widget, and I think even suggests a sensible default (the next working day), when you do payments through their online bank. For the longest time they left the user with a bare bones YYYYMMDD text input widget.

Booking tickets on the web is often a series of frustrations for me. This is in spite of the fact that I am a skilled web user. For example a form won't submit and the validation error is hardly visible. The user is left wondering if the form submitted at all, if there was a validation error, and if so, where?

When I am booking tickets I am almost always in a hurry. Buying a ticket shouldn't take more than a few minutes, and it should be a simple and streamlined process. You want to reduce complexity, minimize the number of options and thus the likelihood of the user being confused.

When ordering my tickets I chose to become member of a frequent flyer program. I checked my email and found about five different confirmation emails from the airline. One of them contained my account number which I eventually found, only I couldn't use that number to log in, I had to figure out myself that I needed to strip the first two characters (leaving only digits) for it to work. I then had to dig out my password, I think from a different email. Once I had logged in I was offered to change my password. Only I couldn't use my standard password that I have memorized, I was forced to choose a new four digit password that I may end up forgetting.

In the midst of usability shortcomings one can of course always take comfort in the fact that online bookings are still better than their offline alternatives. We still have a long way to go with web usability, and as usual, I think most of the answer lies in embracing simplicity. It has struck me again and again through my different endeavours - programming, writing, UI design - that simplicity is key. With simplicity comes focus and a clear goal. Embracing simplicity is something most people can agree with in principle, but putting it to practice takes courage. It is all too easy to go with the main stream complex solution, it seems so much more sophisticated and professional and makes you feel competent and special. After all, you are one of the chosen few who can understand the solution. Therein lies the problem.

2006-11-27T01:02:00.000Z

Third Place in Salsa Competition

There is a quote by Woody Allen that rings more and more true to me:

Eighty percent of success is showing up

Last friday me and my girlfriend Janne were able to win third price at a Salsa competition at La Isla here in Stockholm. Thanks to the encouragement of Pilar and Ociel at Stockholm Salsa dance we decided to give the competition a try and it was great fun! My friends Håkan and Mimmis won the silver, and a very professional and choreographed couple from Uppsala took home the gold. We were very pleased to walk home with the trophies and 1000 SEK though...

There is a huge difference between social dancing and show dancing, and as my friend Håkan said, if you want to be a successful show dancer you need to learn to dance outwards (with the audience) rather than inwards (with yourself and your partner and the music).

2006-11-15T23:08:00.000Z

Teaching Rails at Great Works

I've started teaching Ruby on Rails at Great Works and I'm very excited about it! It's a three evening course given to a small group of up to 10 people. We use a lot of hands on excercise rather than lecturing and the goal is to build a small Rails application. The course participants are designers rather than programmers so we needed to lay down some object orientation and programming basics before we could take on Rails. Despite of this obstacle we were able to make a lot of progress by using Locomotive for installation and scaffolding for UI generation. You can check out the slides here.

I genuinely enjoy teaching and am now looking for more opportunities to help companies learn and get started with Rails quickly. Get in touch if this sounds interesting.

2006-11-15T14:23:00.000Z

Stockholm Rails Meetup Tonight was Awesome

We had a fantastic turnout and great atmosphere at the Rails Meetup at Valtech here in Stockholm tonight. There was a great presentation by Ola Bini on web services and Rails, and one by Christian and Albert at Adocca about caching and a bunch of plugins that they have realeased at RubyForge. All very exciting.

I gave a presentation on the Media Contact Manager system that I have built for Green Media Toolshed.

2006-11-14T01:32:00.000Z

Rails: File Uploading and Storage

File uploading and storage is no big deal, right? I mean with Ruby it can be as easy as:

  def create
    File.open("/Users/db/_dev/rails/videos/swing.mov", "w") do |f|
      f.write(@params["video"].read)
    end
  end

Don't forget to set :multipart => true on your form though. The easiest (and most naive?) approach to uploading files is probably to use the File Column plugin. The Rails Wiki has a good page that discusses file uploads in more depth. The file system base storage approach that I've used in my Rails app is similar to what's described here. I use the after_save and after_destroy callbacks to create and delete the file in the file system - an approach also described in this Rails manual. Johan Sorensen has some code for doing buffered file uploads that might be useful for huge files.

This page is quite impressive. It has everything you ever wanted to know about the input type="file" form widget, including why browsers will ignore the value attribute if you try to set it. An annoyance is that if the user gets validation errors on a form containing a file_field widget she will have to re-select the file to upload.

The classic controversy is whether to store files in the database or in the filesystem. If you store them in the database you have a more unified and consistent storage solution. You keep all your data in one place, in one format, and the only data you need to back up is your database. On the other hand, putting files in the file system could potentially offload your database and make the database dumps smaller. Also, databases were not really intended to store files, that's what file systems are for. Personally I'm ambivalent on this issue, but due to some incidents with file system storage recently I'm now leaning more in favor of storing files in the database.

2006-11-11T03:40:00.000Z

Rails script/console - Did you know it could do all this?

Yesterday someone pointed out some features of the Rails console that I had missed out on. Here are some objects and commands to be aware of:

2006-11-09T14:11:00.000Z

Rails Patch: Rake :test should not fail silently

Swallowing exceptions and failing silently has to be one of the worst sins you can commit as a programmer and I was surprised to run into that issue with the Rake :test task. The Rake :test task (the default task) will in turn invoke the tasks test:units, test:functionals, and test:integration. Any exceptions raised within these sub tasks are not exposed directly but are merely noted as an overall test failure - obscuring the root cause of the failure. This is obviously not very debugging friendly if you have an exception raised somewhere deep down in the Rake task dependency hierarchy. I submitted a patch to remedy the problem.

2006-11-09T13:57:00.000Z

Rails QA: Watch Out for Duplication in your Fixtures

I had a long debugging session due to duplicated record keys in one of my fixture files. It turns out the YAML parser will not complain if you have duplicated record keys (the keys on the top level) or duplicated column keys. Eventhough the YAML specification says that map keys should be unique the YAML parser will happily overwrite any existing value for a key if it encounters that key again.

Under the motto of crashing early and avoiding silent failures I came up with the following unit test:

# Check for duplication in Fixture files, i.e. that:
#
# - keys of records are unique
# - column names are unique
# - id values are unique
#
# The idea is that this test can complent the syntax checking of the YAML
# parser and help avoid debugging nightmares due to duplicated records or columns.
# What the YAML parser will do when a record or column key is duplicated is it
# will just use the last one and let that overwrite the earlier ones.
# Before writing this script I tried using the Kwalify YAML validator but I
# couldn't quite coerce it into doing what I wanted.
class FixtureTest < Test::Unit::TestCase
  def test_fixtures
    fixture_file_paths.each do |file_path|
      initialize_variables(file_path)
      fixture_contents(file_path).each do |line|
        next if skip_line?(line)
        if is_record?(line)
          assert_record_not_dupe(line)
        elsif is_column?(line)
          assert_column_not_dupe(line)
          assert_id_not_dupe(line) if is_id?(line)
        end # End if statement
      end # End line loop
    end # End fixture file loop
  end

  private
  def initialize_variables(file_path)
    @file_path = file_path
    @record_keys = []
    @column_keys = []
    @ids = []   
  end

  def fixture_file_paths
    Dir["#{Test::Unit::TestCase.fixture_path}/**/*.yml"]
  end

  def fixture_contents(file_path)
    ERB.new(IO.read(file_path)).result
  end

  # Skip YAML directive, comments, and whitespace lines
  def skip_line?(line)
    line =~ /^\s*$/ || line =~ /^\s*#/ || line =~ /^---/    
  end

  # A record has no indentation (leading white space)
  def is_record?(line)
    line =~ /^\S/
  end

  # A column line has some indentation (should be same as the first column)
  # and a colon
  def is_column?(line)
    return false if line !~ /^\s+[a-zA-Z_]+:/

    # Looks like a column - check the indentation level as well
    indent_level = line[/^(\s+)/].length
    if @column_indent
      # If the indentation level is different from the first column this may
      # not be a column, it could be nested data of some sort
      return @column_indent == indent_level
    else
      # This is the first column so remember its indentation level
      @column_indent = indent_level
      return true
    end
  end

  def is_id?(line)
    column_key(line) == "id"
  end

  def column_key(line)
    line[/^\s+([^:]+):/, 1]
  end

  def assert_record_not_dupe(line)
    record_key = line[/^(?:- )?([^:]+):/, 1]
    assert !@record_keys.include?(record_key),
      "Record key #{record_key} in fixture file #{@file_path} " +
      "is duplicated on this line: #{line.chomp}"
    @record_keys << record_key
    @column_keys = []
    @column_indent = nil
  end

  def assert_column_not_dupe(line)
    assert !@column_keys.include?(column_key(line)),
      "Column #{column_key(line)} for record #{@record_keys.last} is duplicated " +
      "in file #{@file_path} on this line: #{line.chomp}"    
    @column_keys << column_key(line)
  end

  def assert_id_not_dupe(line)
    id_value = line[/^\s+id:\s*(\S+)/, 1]
    assert !@ids.include?(id_value),
      "Value for id column duplicated in file #{@file_path} on this " +
      "line: #{line.chomp}"
    @ids << id_value
  end
end
2006-08-24T06:53:00.000Z

Rails Recipe: Checking for Broken Links in Integration Tests

Having a way to automatically check for broken links in your application can be quite valuable and there is a simple way to do that with integration tests. Here is the DSL methods that I used for that purpose along with some sample usage:

  module TestingDSL
    def visits_start_page      
      get_with_links("/mcm") do
        assert_tag :tag => 'a', :attributes => {:href => '/mcm/contacts/search_form'}
      end
    end

    def get_with_links(*args)
      request_with_links(:get, *args)
    end

    def post_with_links(*args)
      request_with_links(:post, *args)
    end

    def request_with_links(method, *args)
      self.send(method, *args)
      yield if block_given?
      assert_response :success
      check_links            
    end

    def check_links
      find_all_tag(:tag => 'a').each do |anchor|
        url = anchor.attributes['href']
        if check_url?(url)
          get(url)
          assert_response_code
        end
      end      
    end

    def check_url?(url)
      [/mailto:/, /javascript:/, /user\/logout$/, /:\/\//, /^\#$/].each do |skip_pattern|
        return false if url =~ skip_pattern
      end
      true
    end
    
    def assert_response_code
      assert [200, 302].include?(@response.response_code), 
        "Invalid response code #{@response.response_code} for path #{@request.path} #{@response.body}"
    end
  end

I've found this approach to link checking helps me find bugs that other tests don't find. The only downside is that it can significantly increase the number of requests in your integration tests and thus the test time. There are ways to work around that though, you don't have to check links on every page request, and the link checker could be extended to not request the same URL twice.

2006-08-23T08:37:00.000Z

Rails Workaround: Preserving Nested Hash Params With List Values Across Requests

Suppose you have an HTML search form with a multi-valued select with the name "person[personal][interests][]". The trailing brackets are there to indicate to rails that this HTTP parameter should be converted to a Ruby list. Now, suppose the form submits to the search action in the contacts controller. Our params hash will then contain:

    params = {
      :controller => 'contacts',
      :action => 'search',
      :person => {
        :personal => {
          :interests => ['music', 'chess']
        }
      }
    }

So far so good. Now suppose you have pagination links on the search results page and also that you want to provide a link or button back to the search form for refining the search criteria. Here come the bad news - url_for which is used to create links doesn't support nested hash parameters and also doesn't support list values (there are some patches already to address at least the nesting problem). Here is what url_for produces in this case:

    > puts CGI.unescape(url_for(params))
    > /contacts/search/?person=personalinterestsmusicchess

Clearly url_for is inadequate in this case. I have written a set of helper methods to help save the day:

  def flatten_hash(hash = params, ancestor_names = [])
    flat_hash = {}
    hash.each do |k, v|
      names = Array.new(ancestor_names)
      names << k
      if v.is_a?(Hash)
        flat_hash.merge!(flatten_hash(v, names))
      else
        key = flat_hash_key(names)
        key += "[]" if v.is_a?(Array)
        flat_hash[key] = v
      end
    end
    
    flat_hash
  end
  
  def flat_hash_key(names)
    names = Array.new(names)
    name = names.shift.to_s.dup 
    names.each do |n|
      name << "[#{n}]"
    end
    name
  end
  
  def hash_as_hidden_fields(hash = params)
    hidden_fields = []
    flatten_hash(hash).each do |name, value|
      value = [value] if !value.is_a?(Array)
      value.each do |v|
        hidden_fields << hidden_field_tag(name, v.to_s, :id => nil)          
      end
    end
    
    hidden_fields.join("\n")
  end

Here is the output of my helpers for our example hash above:

  > puts CGI.unescape(url_for(flatten_hash(params)))
  > /contacts/search?person[personal][interests][][]=music&person[personal][interests][][]=chess

  > puts hash_as_hidden_fields(params)
  > <input name="action" type="hidden" value="search" />
  > <input name="controller" type="hidden" value="contacts" />
  > <input name="person[personal][interests][]" type="hidden" value="music" />
  > <input name="person[personal][interests][]" type="hidden" value="chess" />

Hope this will save other Rails programmers some time and frustration.

2006-08-18T05:38:00.000Z

Rails Recipe: HTML Validation

In this howto I'll show a simple approach to HTML validation that I use in my current Rails application. For me, HTML validation is a way to achieve wide browser compatibility, and to do a baseline check for correct rendering and UI brokenness. By putting ampersands and less-than signs in my test fixtures I can use HTML validation tests to check that I haven't forgotten any HTML quoting in my templates, i.e. that I haven't forgotten to use the following constructs:

<%=h some_variable %>
<%= link_to h(some_variable) ... %>

Two common tools for HTML validation are the W3C Validator and Tidy and since I've found them to be complementary I've decided to use both. Tidy warns about empty tags which the W3C validator doesn't. On the other hand Tidy sometimes misses obvious errors such as missing paragraph end tags.

The approach that I came up with for HTML validation was to do it in an after_filter, but only when tests are run, so I added the folling to my test_helper.rb:

# HTML validate the response of all requests
require File.join(File.dirname(__FILE__), '..', 'app', 'controllers', 'application')
class ApplicationController
  after_filter :assert_valid_markup

  def status_code
    @response.headers['Status'][0,3].to_i
  end

  def assert_valid_markup
    return if RAILS_ENV != 'test'
    return if !(status_code == 200 &&
      @response.headers['Content-Type'] =~ /text\/html/i && @response.body =~ /<html/i)

    assert_tidy

    # Going to the W3C validator over HTTP is a bit slow so we make this optional
    return if !ENV['HTML_VALIDATE']
    assert_w3c_validates
  end

  def assert_tidy
    tidy = RailsTidy.tidy_factory
    tidy.clean(@response.body)

    unless tidy.errors.size.zero?
      message = ("-" * 40) + $/
      i = 1
      @response.body.each do |line|
        message << sprintf("%4u %s", i, line)
        i += 1
      end
      message << ("-" * 40) + $/
      message << tidy.errors.join($/)
    end
    raise "Tidy detected html errors in response body: #{$/} #{message}" unless tidy.errors.size.zero?
    tidy.release
  end
 
  def assert_w3c_validates
    require 'net/http'
    print "Querying W3C XHTML validator ... "
    response = Net::HTTP.start('validator.w3.org') do |w3c|
      query = 'fragment=' + CGI.escape(@response.body) + '&output=xml'
      w3c.post2('/check', query)
    end
    raise response.body if response['x-w3c-validator-status'] != 'Valid'
    print response['x-w3c-validator-status']   
  end
end

As you can see from the code above, I only do the time consuming HTTP request to the W3C validator if the environment variable HTML_VALIDATE is set. This way I can easily turn off W3C validation, the obvious risk here being that it always stays turned off. Possible solutions include running the tests with full HTML validation nightly and to install the W3C validator locally.

In the code above you can also see that I use the excellent assert_tidy command from the RailsTidy plugin, so installing that plugin along with the Tidy library itself is a prerequisite for the code to work.

Great tools that I use for manual HTML validation include the Web Developer Extension for Firefox with its W3C validation capability for local HTML and the Safari Tidy plugin that allows you to see for every loaded page any Tidy errors and warnings.

When selecting a DOCTYPE to validate against I was choosing between XHTML 1.0 strict and transitional and I was convinced by certain experts that strict was the way to go.

Doing HTML validation is not without its frustrations of course. For example I had to work around the fact that in XHTML 1.0 strict, form elements (input, select etc.) need to be inside a p, div, or fieldset tag. Also, Tidy requires tables to have the summary attribute. Those are just small annoyances though and I haven't come across any bigger stumbling blocks yet. All in all I'm very happy about my validation efforts and I have a lot more confidence in the UI of my Rails application now that I'm validating its markup automatically in my controller and integration tests.

2006-08-17T04:17:00.000Z

Rails Custom Validation

Suppose you have validation code for a certain type of data that you want to reuse across your ActiveRecord models. What is the best approach to doing this in Rails? As you probably know, Rails comes with a number of validation macros such as validates_presence_of, validates_format_of, and validates_uniqueness_of etc. One approach is to write your own validation macro that wraps one of the Rails validation macros. The validates_as_email plugin shows us how:

module ActiveRecord
  module Validations
    module ClassMethods
      def validates_as_email(*attr_names)
        configuration = {
          :message   => 'is an invalid email',
          :with      => RFC822::EmailAddress,
          :allow_nil => true }
        configuration.update(attr_names.pop) if attr_names.last.is_a?(Hash)

        validates_format_of attr_names, configuration
      end
    end
  end
end

The RFC822::EmailAddress constant in the code above is a huge regexp that I'm not going to list here... A macro that wraps validates_format_of works well as long as your validation can be done with a regexp. What about validations that are too unwieldly or impossible to do with a single Regexp? Suppose you want to validate phone numbers to make sure they have 10 digits and only contain the characters 0-9()/-.+. Using validates_format_of as a starting point we could write our own macro:

      def validates_as_phone(*attr_names)
        configuration = {
          :message => 'is an invalid phone number, must contain at least 5 digits, only the following characters are allowed: 0-9/-()+', 
          :on => :save 
        }

        validates_each(attr_names, configuration) do |record, attr_name, value|
          n_digits = value.scan(/[0-9]/).size
          valid_chars = (value =~ /^[+\/\-() 0-9]+$/)
          if !(n_digits > 5 && valid_chars)
            record.errors.add(attr_name, configuration[:message])
          end
        end
      end

In the general case we could reduce duplication by hacking validates_format_of and make its :with option accept Proc objects:

      def validates_format_of(*attr_names)
        configuration = { :message => ActiveRecord::Errors.default_error_messages[:invalid], :on => :save, :with => nil }
        configuration.update(attr_names.pop) if attr_names.last.is_a?(Hash)
        
        raise(ArgumentError, 
          "A regular expression or Proc object / method must be supplied as the :with option of the configuration hash") \
            unless (configuration[:with].is_a?(Regexp) || validation_block?(configuration[:with]))

        validates_each(attr_names, configuration) do |record, attr_name, value|
          if validation_block?(configuration[:with])
            value_valid = configuration[:with].send('call', value)
          else
            value_valid = (value.to_s =~ configuration[:with])
          end
          record.errors.add(attr_name, configuration[:message]) unless value_valid
        end
      end
      
      def validation_block?(validation)
        validation.respond_to?("call") && (validation.arity == 1 || validation.arity == -1)
      end

Using our new validates_format_of could look like:

  validates_format_of :phone, :fax, :with => Mcm::Validations.valid_phone


  module Mcm
    class Validations
      def self.valid_phone
        Proc.new do |number|
          return_value = false
          if !number.nil?
            n_digits = number.scan(/[0-9]/).size
            valid_chars = (number =~ /^[+\/\-() 0-9]+$/)
            return_value = (n_digits > 5 && valid_chars)
          end
          return_value
        end
      end    
    end
  end

An issue with this approach is that the validation proc object may want to determine what the error message should be. What we could do is adopt the convention that if the Proc object returns a string, then that is the error message.

After all this touching of interal Rails method (potentially making us vulnerable to upgrades), we arrive at the safer and simpler approach that Rails provides for custom validations, namely to implement the validate method:

  # In the ActiveRecord class to be validated
  def validate
    Mcm::Validations.validate_phone(self, 'phone', 'fax')
  end

  # In lib
  class Mcm::Validations
    def self.validate_phone(model, *attributes)
      error_message = 'is an invalid phone number, must contain at least 5 digits, only the following characters are allowed: 0-9/-()+'
      attributes.each do |attribute|
        model.errors.add(attribute, error_message) unless Mcm::Validations.valid_phone?(model.send(attribute))
      end
    end    

    def self.valid_phone?(number)
      return true if number.nil?

      n_digits = number.scan(/[0-9]/).size
      valid_chars = (number =~ /^[+\/\-() 0-9]+$/)
      return n_digits > 5 && valid_chars
    end  
  end

  # In config/environemnt.rb (not sure why this is needed)
  require 'validations'

We can make the validate method approach more Rails like by using a mixin instead:

  # In the ActiveRecord class to be validated
  def validate
    validate_phone('phone', 'fax')
  end

module Mcm::Validations
  def validate_phone(*attributes)
    error_message = 'is an invalid phone number, must contain at least 5 digits, only the following characters are allowed: 0-9/-()+'
    attributes.each do |attribute|
      self.errors.add(attribute, error_message) unless valid_phone?(self.send(attribute))
    end
  end    

  def valid_phone?(number)
    return true if number.nil?

    n_digits = number.scan(/[0-9]/).size
    valid_chars = (number =~ /^[+\/\-() 0-9]+$/)
    return n_digits > 5 && valid_chars
  end  
end

class ActiveRecord::Base
  include Mcm::Validations
end

Which looks clean and I'm reasonably happy with this last approach. Finally...

2006-08-16T04:48:00.000Z

Rails Recipe: A Timezone Aware Datetime Picker

NOTE! Only follow this recipe if you are using an earlier version of Rails than 2.1. As of Rails 2.1 timezone support is built into Rails in a way that makes it much easier to create timezone aware applications.

Quite often in web applications we display dates and times and also have users input them and store them in our databases. If your users are spread across the globe you really want to be able display times in the users own timezone. How do we accomplish this with Rails?

First off, as Scott Baron has pointed out, you want to grab a copy of the TZInfo Ruby timezone library. We need TZInfo since it can deal with daylight savings time (summer/winter hourly adjustments), something that the Timezone library that ships with Rails is not able to do. Installation is as simple as downloading the latest tgz file from RubyForge, extracting it into vendor/tzinfo, and requiring the library from config/environment.rb. We also add some new settings:

# In config/environment.rb
ActiveRecord::Base.default_timezone = :utc # Store all times in the db in UTC
require 'tzinfo/lib/tzinfo' # Use tzinfo library to convert to and from the users timezone
ENV['TZ'] = 'UTC' # This makes Time.now return time in UTC

The strategy we are adopting here is that times in the UI when shown to or entered by the user are in the users own timezone. In the application on the other hand, i.e. in Ruby code and in the database, times are always kept in the UTC timezone. Our job then becomes to allow the user to select a timezone, and then to convert back and forth between this timezone and the UTC timezone as needed. To store the user timezone we can add a time_zone string column to the users table and use the composed_of macro just like Scott Baron describes:

class User < ActiveRecord::Base
  composed_of :tz, :class_name => 'TZInfo::Timezone', 
              :mapping => %w(time_zone time_zone)
end

Then add a timezone select on a "Set Timezone" preference page that the user can access:

<% # Select using TZInfo timezone names such as "Europe - Amsterdam"
   # In the controller on submit you can then do 
   # @user.tz = TZInfo::Timezone.new(params[:user][:timezone_name])
<%= time_zone_select 'user', 'timezone_name', TZInfo::Timezone.all.sort, :model => TZInfo::Timezone %>

If you prefer the timezone names of the Rails Timezone class you can use them instead and then convert to a TZInfo timezone object:

<% # Select using the Rails Timezone names such as "(GMT+01:00) Amsterdam". Will require a
   # conversion to the TZInfo timezone on submit. %>
<%= time_zone_select 'user', 'timezone_name' %>

# Helper method in the controller
def tzinfo_from_timezone(timezone) 
  TZInfo::Timezone.all.each do |tz|
    if tz.current_period.utc_offset.to_i == timezone.utc_offset.to_i
      return tz
    end
  end
  return nil   
end

# On submit in the controller we convert from Rails Timezone to TZInfo timezone via the UTC offset
# and store the user timezone in the database.
@user.tz = tzinfo_from_timezone(TimeZone.new(params[:user][:timezone_name])
@user.save

Now when displaying times in the UI we can consistently convert them to the users timezone with a helper like this:

  def format_datetime(datetime)
    return datetime if !datetime.respond_to?(:strftime)
    datetime = @user.tz.utc_to_local(datetime) if @user
    datetime.strftime("%m-%d-%Y %I:%M %p")
  end

To have the user enter a date and a time in a user friendly fashion you can install the bundled_resource plugin and use its JavaScript based calendar date picker. When displaying the date to the user in an HTML form we use @user.tz.utc_to_local to convert from UTC to the users timezone, and when receiving a date from a form submit we convert back to UTC with @user.tz.local_to_utc:

# The new action:
def new
  @email = BulkEmail.new
  @email.schedule_date = @user.tz.utc_to_local(Time.now) # Default schedule date in local time
end

# new.rhtml
<%= dynarch_datetime_select('email', 'schedule_date', :select_time => true) %>

# The create action that the new form submits to
def create
  @email = BulkEmail.new(params[:email])
  # Convert the local schedule date from the form to UTC time
  @email.schedule_date = @user.tz.local_to_utc(@email.schedule_date)
  if @email.save
    ...
  else
    ...
  end
end

# The edit action
def edit
  @email = BulkEmail.find_by_id_and_group_id(id, session[:group_id])
  # Show scheduled date in local time
  @email.schedule_date = @user.tz.utc_to_local(@email.schedule_date) 
end

# The update action that edit submits to
def update
  @email = BulkEmail.find_by_id_and_group_id(id, session[:group_id])
  @email.attributes = params[:email]
  @email.schedule_date = @user.tz.local_to_utc(@email.schedule_date)

  if @email.save
    ...
  else
    ...
  end  
end

Apparently there is also a Ruby on Rails TZInfo plugin that I discovered only now as I was writing this post and I haven't looked into using it yet. A very helpful page when dealing timezones is the timeanddate.com World Clock.

Testing of the functionality described here is left as an exercise for the reader...

2006-08-16T01:08:00.000Z

Rails Recipe: CSV Export

A common requirement from customers is the ability to export tabular data to a CSV file that can be imported into Excel. Ruby on Rails uses the standard Ruby CSV library to import test fixtures that are in CSV format. However, there is a runner up CSV library called FasterCSV which as the name suggests is faster and also has a much cleaner API. I was hesitant to use FasterCSV in production since it is still in Beta but after finding the discussion about including FasterCSV in the Ruby standard library I decided to give it a try.

The first step was to download the latest FasterCSV tgz file and extract into RAILS_HOME/vendor/fastercsv. I then added the following line to config/environment.rb:

require 'fastercsv/lib/faster_csv'

The action that exports the CSV data looks roughly like the following:

  def csv
    @list = @group.lists.find(params[:id])

    csv_string = FasterCSV.generate do |csv|
      csv << ["Type", "Name", ... more attribute names here ... ]

      @list.records.each do |record|
        csv << [record['type'],
                record['name'],
                ... more values here ...]
      end
    end

    filename = @list.name.downcase.gsub(/[^0-9a-z]/, "_") + ".csv"
    send_data(csv_string,
      :type => 'text/csv; charset=utf-8; header=present',
      :filename => filename)
  end

In the corresponding controller test I check that the CSV response can be parsed with the other CSV parser, that there is a header row with expected attributes, that the number of data rows is correct, and finally that one of the rows has the correct values:

  def test_csv
    get_success :csv, :id => lists(:peterlist).id

    reader = CSV::Reader.create(@response.body)

    header = reader.shift
    ['Type', 'Name', ... more attributes here ...].each do |attribute|
      assert header.include?(attribute)
    end

    rows = []
    reader.each do |row|
      rows << row
    end
    
    assert_equal rows.size, lists(:peterlist).size
    item = lists(:peterlist).items.first
    item_row = rows.select { |row| row[1] == item.contact.name }.first
    assert_equal item_row[0], 'Contact'
    assert_equal item_row[1], item.name
    ... assertions for the other attribute values here ...
  end

To see what CSV export from a controller looks like with the Ruby CSV library - check out this blog post.

2006-07-18T22:58:00.000Z

This Might Cost You a Little...

I just received an invoice from my dentist for 900 SEK (about a 100 EUR) for some unspecified service. I called them and it turned out I was paying 900 SEK for being 15 minutes late. What happened was I over slept and four minutes into the appointment when I was rushing out of my apartment to get on my bike my dentist calls (from Folktandvården, Eastmaninstitutet). I apologize and tell her I can be at her clinic in 7 minutes on my bike. She says nah, I'm not sure it's worth you coming in now that you're late, we better schedule a new time for you. I say ok, given that I didn't seem to have a choice. We book a new time and then at the end of the conversation she mentions that "oh, this might cost you a little" (the exact words in swedish were "det här kommer kanske att kosta lite").

Interesting points of comparison here is that once before when I missed an apointment I paid 300 SEK. My mom says she has never had to pay for missed appointments at her dentist. Last time I visited the dentist and they examined my teeth I paid 500 SEK, which right now seems pretty cheap, especially given that that time they actually performed a service for me.

2006-07-04T13:04:00.000Z

The Weepies Make me Weep with Joy

I just came home from a concert with The Weepies (a folk/pop duo from the US) at Debaser here in an exceptionally warm and sunny Stockholm.

The Weepies made me all misty eyed, showering beautiful harmonies of vocals and guitars over a small audience in an intimate concert setting. I loved the fact that about half of the songs were new to me since they were drawn from the solo CDs of Deb Talan and Steve Tannen that preceeded The Weepies. They also played a new song about an "Old Coyote" to be released soon on Itunes...

I wish there would have been more people to cheer Deb and Steve and Meghan Toohey (guitar) on. The band was so humble and appreciative of the few of us who had shown up though that it didn't seem to be a problem. Afterwards I stayed around and chatted a bit with Deb, Steve, and Meg and they are absolutely lovely, warm, and down to earth people.

I discovered The Weepies by chance last christmas holiday thanks to a "Staff Favorites" recommendation in Itunes Music Store. In a way The Weepies made me rediscover and re-appreciate pop music, and that's no mean feat. Over the last years I've been so busy listening to and dancing to salsa music (and some jazz and tango on the side) that I haven't been able to keep up with mainstream pop much.

Do check out The Weepies on Itunes Music Store and don't miss the cute story of how they met. It's almost too perfect to be true.

The Weepies will make it big. I just know it. Next time they play in Stockholm they will probably be way too popular for me to be able to go and talk to them after the concert, unfortunately. Thanks for making it all the way here to play - and welcome back!

2006-06-07T22:43:00.000Z

Rails Code Generation Considered Useful

I just checked out Ryan Daigle's excellent What's New in Rails blog. There are a lot of great findings there for anyone like me who hasn't monitored closely the changes to the Ruby on Rails source code. In the post about the observer generator Ryan writes:

"On a side note, I feel like the generator utility has been getting a bad rap recently from people that frown upon auto-generation of code. Fair enough, but think of them more as best practices templates and learning tools rather than some sort of coding robot. They're really great tools for seeing how things should be done, and how things are expected to be done. If you don't want to use them, don't - but if you're not quite sure how to go about creating an observer, or controller, or model they're of great help."

Ryan puts it very well and I couldn't agree more. I've found the scaffold generator very useful in the project I'm working on now, and it's not for prototyping I'm using it, but for providing a skeleton for the production code I'm going to write.

Another advantage of the code generation in Rails is that if you use it your code will probably follow conventions (naming of controller actions for example) more closely and this will make the code easier to read and maintain for other developers. The generated code also contains best practices (such as the verify :method => :post for certain actions in controllers) that an average Rails programmer (or even a good Rails programmer) may not know about or may forget to use otherwise.

2006-05-10T03:18:00.000Z

Presentations from the Rails Recipes Meetup

The Rails Recipes Meetup yesterday was a great success with over 20 participants and four interesting presentations. I would like to thank Xlent for hosting us and all the people who showed up and contributed to making the evening so nice.

If you are interested in coming to the next meetup (scheduled to happen in June or July), please signup for the swedish Rails mailing list at rails.se. Please also suggest presentation topics on that mailing list.

Here are two of the presentations from the meetup:

See you at the next meetup!

2006-04-29T07:01:00.000Z

Ruby on Rails Tips and Tricks

I've started a wiki page with some Ruby on Rails tips and tricks that I pick up from various weblogs and books. Hope it might be useful to others as well.
2006-04-26T21:44:00.000Z

Starting a Startup - Hard but Doable

Here are some encouraging words from Paul Graham's How to Start a Startup article (which will make you scroll through eternity):

"You need three things to create a successful startup: to start with good people, to make something customers actually want, and to spend as little money as possible. Most startups that fail do it because they fail at one of these. A startup that does all three will probably succeed.

And that's kind of exciting, when you think about it, because all three are doable. Hard, but doable. And since a startup that succeeds ordinarily makes its founders rich, that implies getting rich is doable too. Hard, but doable."

Interesting to see the Web 2.0 Startup Toplist as well for a list of successful startups.

From Paul Graham's Y Combinator:

"Y Combinator relies on certain premises: that open-source software and falling hardware prices means that tech start-ups are cheap to finance; that large companies are no longer at the forefront of innovation; and that mature technology companies find it cheaper to buy than to build."

2006-04-26T20:04:00.000Z

Rails Recipes Meetup in Stockholm

Each presentation is up to 20 minutes long followed by a 10 minute discussion. The presentations should be in the Rails tradition of "show don't tell". We prefer short presentations with code samples from recipes applied to real problems over the traditional type lengthy and theoretical Powerpoint presentations.

"Now, I know Rails well enough that over time I may have stumbled on some of the solutions, but that's not the point. With a collection of great recipes at hand - In-Place Form Editing, Live Search, RJS Templates, Polymorphic Associations, Tagging, Syndicate Your Site with RSS, and Sending Email with Attachments to name a few - I can quickly give my app the functionality today's web users have come to expect, then move on to the features that really set it apart from the crowd."

2006-04-07T00:52:00.000Z

Quote of the Day

Thanks Jarkko for highlighting the following nice quote from the article Two more reasons why so many tech docs suck which argues the case of getting your users excited and using use cases for technical documentation:

One of the best techniques for creating and keeping interest is to make non-reference docs use-case driven. That way, each topic is framed by a context that matches something the user really wants to do. That way, each little sub-topic shows up just-in-time, instead of appearing to be there just-in-case.

It makes a lot of sense to me to organize technical documentation by the readers goals, i.e. to use the howto or cookbook approach. The Perl Cookbook is a great example of how this approach can be successful. In that instance the use-cases are on a very low level such as "How do I iterate over an array" but the approach should also work on a higher level, i.e. "How do I setup a weblog". High level use-cases like that are probably best broken down into smaller use-cases.

2006-03-31T02:40:00.000Z

Quote of the Day

I was catching up with a lot of Rails related posts in my RSS reader and found DHH's bashing of James McGovern which was pretty entertaining. I also like this quote in the comments:

First they ignore you
Then they laugh at you
Then they compete with you
Then you win

I also learned today that Dreamhost upgraded to Rails 1.1. Apparently my homepage survived that upgrade. I still don't have Rails in the vendor directory but I do have web tests set up so I don't have to worry too much.

2006-03-21T13:13:00.000Z

Rails Meetup @ Akkurat

The Akkurat pub

Tonight a bunch of hardcore geeks here in Stockholm met up at the excellent Akkurat pub to chat about Ruby and Rails. More than Rails though, the conversation seemed to drift towards ABC 80, Atari, and Amiga nostalgia which was great fun. I am very positively surprised by the atmosphere and service at Akkurat by the way and I now consider it to be my favorite pub here in Stockholm. The next time I need to try their special "Heaven" and "Hell" beers.

I've thoroughly enjoyed both the Rails meetups that we've had so far. The first one was held at The Dancing Dingo, the second one tonight at Akkurat, and the third one we are planning to have in a couple of weeks at Saddle and Sabre. To signup for notifications about future meetups in Stockholm make sure to get on the swedish Rails mailing list which you'll find at rails.se.

2006-03-19T06:30:00.000Z

Sunday Brunch with a Touch of Flair

Stylish lobby of Hotel Amaranten

I enjoyed an exotic brunch today with my girlfriend at the flairful Amaranten hotel here on Kugsholmen in Stockholm. As you can see, the hotel lobby and restaurant is stylish and has a nice ambience. The buffet is offered at 160 SEK and is high quality with a good selection of meat such as chicken wings, spare ribs, bacon, and a number of more exotic dishes that I don't even know the name of :-) Some of the Cajun specialties didn't particularly speak to our taste but all in all it was quite good. On the minus side was the fact that the place was almost empty and one of the waitresses was not so service minded.

The atmosphere was enhanced by a house Jazz band - a trio on bass, drums, and piano. They performed professionally albeit somewhat uninspired, which is understandable given the lack of an engaged audience. We tried to cheer the band on by taking the initiative to applaud and I think this was appreciated.

I seem to have taken up an interest in luxury hotels lately - there is something fascinating about them. A couple of years ago I stayed with my girlfriend at the Grand hotel here in Stockholm - a very memorable and highly recommended experience. Recently I read in the paper about how the new Hotel Rival (owned by famed Benny Anderson of Abba) in Stockholm came in highest in scandinavia (at place ninety something) in a listing of the most luxurious hotels in the world. This sparked my interest and I have now booked a brunch there for the second of April and will spend the night with my girlfriend in one of their "de luxe" rooms. It's no exaggeration to say that I look forward to that...

2006-03-07T12:21:00.000Z

Less is More - The 37Signals Mantra

I remember seeing a presentation by Jason Fried of 37Signals at Reboot in Copenhagen a couple of years ago. The presentation was about defensive web design and it was very professional and convincing. It seems now though, Jason has become not only an authority on web design but also on the general process of how to successfully build websites. I listened to Jason's interview at IT conversations a while back and it really impressed me. I felt like listening to the speech several times and write down all the important points that Jason is making. However, 37Signals beat me to that task and for only $19 you can have the Getting Real book which essentially covers and expands on the same philosophy and guidelines that are in the speech. The book notwithstanding, here are my own sketchy notes from the IT conversations interview:

Reduce mass. Physics tells us that the higher the mass of a moving object the more difficult it is to change its direction. Mass can be too many people, hardware, software lockin, and long term contracts. Ability to change is very important in any product development.

Embrace constraints. Examples of constraints are hardware, people, and money. Constraints is where creativity happens. Let people use the product and look for real patterns. Start with less and your more with be a lot more clear down the road. The initial release should be half the product. Everything is beta these days but its time to have confidence, public is public. Keep iterating and make what you have rock solid, bullet proof, and great

Getting real. Use few documents, specs and diagrams. Build top down - build the interface first.

Managing debt. Feel the hurt. The people who built the product should support it. Chefs become waiters once in a while.

Finding people. Look for people who are:

2006-03-07T11:12:00.000Z

My First Acoustic Instrument

I just had to post a picture of my nice new LP Bongo drums:

My Bongo Drums

It was in Copenhagen that I regularly started dancing salsa and through the dancing I developed an interest in the music. On the salsa mailing list in Copenhagen there came an invitation from Jakob Johansen to learn how to play in a salsa band. As a child I always wanted to play the drums but my dad wasn't happy about the idea of having drums in the house so that dream never materialized. However, in Copenhagen I jumped on the opportunity and by chance I ended up playing Timbales in the band. I was the only one in the band who was a total beginner but it worked out ok anyway and I throughly enjoyed it.

Once in Stockholm last year I was looking for the opportunity to continue playing in a salsa band. I found Pygge and Alternativa Musikskolan in Sollentuna where I took private lessons in Timbales. I decided to switch to conga drumming and pursued that for a while. Eventually I found a salsa band that I could play in. However, it turned out they alreay had a conga drummer so I had to learn the bongos instead. I find bongos easier to play than the congas and they are great for soloing. I think my new bongo drums look great - now I just have to learn how to play them...

2006-01-19T04:00:00.000Z

Financial Freedom vs Liking What you Do

Paul Graham writes that:

"Most people would say, I'd take that problem. Give me a million dollars and I'll figure out what to do. But it's harder than it looks. Constraints give your life shape. Remove them and most people have no idea what to do: look at what happens to those who win lotteries or inherit money. Much as everyone thinks they want financial security, the happiest people are not those who have it, but those who like what they do. So a plan that promises freedom at the expense of knowing what to do with it may not be as good as it seems."

Whichever route you take, expect a struggle. Finding work you love is very difficult. Most people fail. Even if you succeed, it's rare to be free to work on what you want till your thirties or forties. But if you have the destination in sight you'll be more likely to arrive at it. If you know you can love work, you're in the home stretch, and if you know what work you love, you're practically there.

2006-01-01T13:25:00.000Z

Lessons from Adam Bosworth

Adam Bosworth's article Learning from THE WEB is well worth reading (thanks Branimir for pointing me to it). A surprising number of the lessons have to do with simplicity, for example the KISS one:

"KISS. Keep it (the design) simple and stupid. Complex systems tend to fail. They are hard to tune. They tend not to scale as well. They require smarter people to keep the wheels on the road. In short, they are a pain in the you-know-what. Conversely, simple systems tend to be easy to tune and debug and tend to fail less and scale better and are usually easier to operate. This isn't news. As I've argued before, spreadsheets and SQL and PHP all succeeded precisely because they are simple and stupid - and forgiving."

Other lessons such as "It is acceptable to be stale much of the time" and "The wisdom of crowds works amazingly well" ring very true to me. His critique of database vendors inability to learn from the web is thought provoking.

2005-12-16T23:28:00.000Z

How I Test my Website

Introduction

This is an article that talks about the motivation for testing and how I went about writing automated tests for my personal website. It is best suited for someone with at least some familiarity with Rails but who hasn't yet explored its testing capabilities. There is also a section on web testing that is independent of Rails.

I learned my lesson in Berlin

I still remember clearly the somewhat embarrassing mistake that sparked my interest in software testing. I was working as an intern for a research institute in Berlin and my task was to program an image processing algorithm in C. Once the program was finished I was stalled for a long time, maybe weeks, puzzled over the unexpected outputs of my program. My supervisor was starting to lose patience with my lack of progress. Eventually it occurred to me that the underlying algorithm that I had implemented was broken due to a trivial mistake. All along I had simply assumed that it was working without conducting any thorough tests to prove my assumption. That's when it dawned on me what a challenging and important problem software testing is.

The miserable past of web testing

A lot of developers see testing as a dull or even unnecessary activity. The level of testing has been particularly poor in web development, a sector of the software industry notorious for being hackish and low in quality. Granted, the cost of a broken web page is typically not as big as a bug in banking software. However, a broken page on a high volume public site or an internet banking site can be very costly indeed.

With Agile methodologies such as Extreme Programming the idea is to make changes, refactor, and release often. Good automated test coverage is almost a prerequisite for this style of development. This is because for each change that we make we need to make sure that we didn't break existing functionality. In a system with high coupling the bug caused by a change can arise in a seemingly unrelated part of the system. Of course, manual testing is not really an option in agile development as it would be too expensive and slow down the development and release cycle too much.

There are benefits to testing other than just finding bugs that are worth mentioning. One is that testing forces you to focus on and define the interface (aka contract) of your application or API. This is the reason why test driven development can be so valuable since defining the contract of your code early on provides a good direction and goal for your programming - it serves as a specification. Also, when you are writing tests you will often find yourself reviewing your code and this review in and of itself can reveal valuable improvements.

Rails shows the way

Most of the hype surrounding Rails has to do with its productivity and the surprisingly few lines of code needed to build a typical website. A nice side effect of having a small code base is that you have fewer lines of code to debug. Testing in Rails is also helped by its strict MVC (Model-View-Controller) structure. Last but not least, Rails comes shipped with a tightly integrated testing framework that allows testing both on the controller and model levels.

Rails has an interesting code statistics tool invoked by "rake stats" that counts the lines of code (LOC) and the ratio of production to test code. Here is the output for my website:

+----------------------+-------+-------+---------+---------+-----+-------+
| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Helpers              |    19 |    18 |       0 |       1 |   0 |    16 |
| Controllers          |   260 |   210 |       6 |      37 |   6 |     3 |
| Components           |     0 |     0 |       0 |       0 |   0 |     0 |
|   Functional tests   |   340 |   253 |      10 |      45 |   4 |     3 |
| Models               |   122 |    91 |       4 |      12 |   3 |     5 |
|   Unit tests         |   191 |   150 |       3 |      17 |   5 |     6 |
| Libraries            |     0 |     0 |       0 |       0 |   0 |     0 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total                |   932 |   722 |      23 |     112 |   4 |     4 |
+----------------------+-------+-------+---------+---------+-----+-------+
  Code LOC: 319     Test LOC: 403     Code to Test Ratio: 1:1.3

Unit testing

A unit test in Rails tests a single model class. For my website I have three model classes - Post, Category, and User. There is a one-to-one mapping in Rails between model classes and database tables. The model classes have methods for the usual CRUD (create, read, update, delete) operations and all of these operations interact with the database.

When you write tests that depend on database data it will simplify your life a lot if the tests have control over that data. This is one of the areas where the Rails testing framework shines. All Rails tests are executed against a separate test database into which data is loaded with the help of fixtures. Each database table has a corresponding fixture YAML file. YAML is a concise and yet readable text data format that is quite handy to work with. The YAML files can contain embedded ruby code and so can be dynamic with loops and variables.

Here are some examples of what my unit tests cover:

Functional testing

In Rails functional tests are applied at the controller level. Each controller class has a corresponding controller test class. The controller test executes actions on its controller, makes assertions about the values of the member variables in the controller, about the type of response (success, redirect etc.), and about the HTML output generated by the view. The controller test also has access to the request and response objects, to the session, and to the flash object used to display status messages to the user. All in all this makes controller tests very powerful.

Here are some functional tests I wrote for my website:

One minor limitation of controller tests to be aware of is that they don't include the URL-action mappings (the routes). One could probably test those separately somehow but I haven't seen an example of that yet. The URL mappings are covered by the web tests though, see below.

There is potential for overlap between unit and functional testing. Thorough testing at the model level decreases (but does not eliminate) the need for testing at the controller level. It is clearly an advantage if the controller tests can rely on the underlying model to be working correctly so that they can focus on the controller specific logic.

Web testing and Monitoring

I use web testing for pre-launch and post-launch acceptance testing and for monitoring of my website. This means I run the web tests just before a site upgrade, immediately after the upgrade, and then regularly every day against the production site. The web tests execute the entire system including the production database. The fact that the production database is used is important since it is not uncommon for bugs to depend on database data. The Rails unit and functional tests can provide good test coverage but the web tests are a necessary complement and are the ultimate check that the system is working at the point of release and that it keeps working in production.

I have divided my web tests into two parts. I have one "ping" test that I run frequently and that just makes sure that the front page works and the site is up. The other test is a smoke test that I run twice a day. For all pages on my site it checks that:

If any of the web tests fail I am alerted by email. I am also alerted by email if an error occurs in the code (i.e. an exception is thrown). With this setup I can rest assured that if major functionality on the site breaks down I will know about it and have a good chance of taking action quickly.

Web testing is not provided by the Rails framework and I have chosen to rely on Perl scripts for those tests. My web tests currently don't do much HTML parsing, but if there was a need for that Perl has its simple yet powerful look_down method. However, if there were similar APIs available in Ruby I would probably switch to Ruby as I am more comfortable with that language.

What's Missing?

There should be some kind of automatic test that verifies that site backups are working. A way to do that would be to have a mirror site in a different location that is updated nightly with the database backup from the production site. One could then run the web tests against the mirror site as well. One would also need to verify, through a web test or otherwise, that no data is missing in the backup. If the production site is not changed during the recreation of the mirror site a sufficient test would be to check if the two sites are identical.

If a website makes heavy use of JavaScript and Ajax then an in-browser testing tool such as Selenium may be useful for acceptance testing and browser compatibility testing.

Last, but not least, if you are expecting a lot of traffic you should look at load testing with tools such as Apache Bench (ab), or siege. Basic usage of these tools is shown in the Rails book. Rails also has the concept of performance tests that make it easy to load huge quantities of data and then repeatedly execute some critical action and measure its average execution time.

Conclusion

The level of testing that I have described here isn't comprehensive and really ought to serve as a baseline for pretty much any serious website. Unfortunately, this is far from the case in the industry today. With the advent of Rails, testing is more readily available to web developers than ever before. Let's remind ourselves to take full advantage of that.

2005-12-06T10:40:00.000Z

The Threat of New Technologies

In the midst of one of the many Java/Rails flamewars (haven't we had enough of them already?) I found a thought provoking comment by the creator of Tapestry on Rails and the resistance to technology change among developers:

"It's natural to be a little scared of new developments. It's painful to think that your exclusive, hard-won skills may be even a little bit obsolete. When I see Flex, Laszlo or Rails, my stomach does a little flip-flop in terms of all the effort I've put into Tapestry. But the reality is that *all* of these approaches are transitory (I'm sure we'll all be doing something quite different in five years), and we should embrace change, learn from other's efforts, and create even better frameworks and applications."

"For example, there may come a time when much Tapestry work is done using Trails (a Tapestry/Rails/Spring/Hibernate hybrid). Trails may never have offer quite the same developer productivity as Rails, but it is still a gateway into full-blown Tapestry and J2EE development."

Openness to new programming languages and toolkits seems to be a common characteristic among leaders in the programming community. The best example of this is probably the Pragmatic Programmers who came up with the now famous advice to learn one new programming language a year. I can't help but wonder that maybe if Dave Thomas hadn't picked up and documented Ruby, David Heinemeier Hansen would not have discovered it either and Rails would not have been a reality today.

I have been impressed that so many Java heavy weights (David Geary/JSF, James Duncan Davidson/Ant, Bruce Tate etc.) have had the courage to step up and recommend Rails. Personally I'm certainly ready and well equipped to make the transition to Rails and the urge to do so is bigger than ever now that I am back working in the less productive and overly complicated Java world.

2005-11-27T09:03:00.000Z

I'm on Rails Now

I was finally able to put my new homepage live late last night after too many hours of programming (why do things always take longer than we expect?). About two weeks ago I caught a cold and I spent several days alone in my apartment. The upside of being sick is that I've had time to work very intensely on my homepage (when I probably should have been resting)...

This is the third incarnation of my homepage. A couple of years ago I started out with an OpenACS site using Lars Pind's weblog software. Since OpenACS hosting is hard to come by and that technology had little momentum I moved to dreamhost.com and Wordpress which both have worked out very, very nicely for me. At that point I didn't know the Rails revolution was coming...

When I was working for Collaboraid with Lars last year we had David Heinemeier Hansson (the creator of Rails) come visit us in our office and tell us about his toolkit. I remember how we were shooting all kinds of tough questions at him of the type "ok, that sounds good, but can Rails do this or that particular thing that we need". David exuded confidence and would patiently and immediately answer each and every one of our questions like "sure you can do that, or in fact, you can do something even better, let me show you...". We were all very convinced. At that time though, I didn't have the time or energy to pick up a new web technology and I was somewhat deterred by the fact that Rails was still such a small niche. Having lived in the OpenACS niche for so long I was very impatient to find my way back into the mainstream.

Rails stayed in the back of my mind though and when I came to Stockholm to work as a Java developer at Pricerunner I noticed how people were talking about Rails. I then realized just how big the technology must have gotten. Over the past months I've read countless articles about Rails and it's really mind boggling to see how Rails had litterally taken the world by storm. I didn't see that coming, at least not at the rate that it's been happening. There is a myriad of open source web toolkits out there competing for attention and people like me who have worked with OpenACS know how hard it is to get a toolkit using an unpopular programming language into the mainstream (there are other more compelling reasons why OpenACS never entered the mainstream, but that's a different story).

When I had realized that the Rails way is really the future of web development, and also had experienced first hand how wonderful the Ruby language is, I knew I had to learn these technologies. I figured that what better way to do that than to build my own homepage using them? I started by trying Typo - the Rails weblogging software that all the other cool Rails hackers seem to be using. However, at the time that I tested it I was kind of put off by some bugs that it had. It seemed awfully immature coming from Wordpress. Also, Rails is not a framework for plugging in and reusing applications (like OpenACS is). Rather, it is a framework for building applications from scratch in a very fast and elegant fashion.

Of course, building any application, even if it's something seemingly simple as a weblog, is going to take a lot of work and thought if you want the result to be top class. What's fascinating is that Rails makes it feasible (because of its productivity) to build from scratch in a lot of cases where previously application reuse was the only realistic option.

I'm really thrilled about web development again know that I know Rails. I'll write up some details about how I built my homepage later on.

My old homepage is still accessible and now lives at old.marklunds.com.

2005-11-25T15:53:00.000Z

Java is not Always the Best Tool? Really?

I really like the article Ruby the Rival over at OnLamp.com, and especially this part by James Duncan Davidson (this has been said many times before - when all you have is a hammer everything looks like a nail):

"And I think that's the real win from the recent attention on Ruby on Rails and the breakaway from viewing the world with Java-colored glasses. It's not that Ruby on Rails is going to be the next Java. Far from it. It's that Ruby on Rails helps to break this idea that there is "One True Way." There's not. There are many different ways to solve a problem. And really, none of them is the clear-cut winner. There's just places where one solution has advantages."

2005-10-30T23:36:11.000Z

PriceRunner is Number One

According to PC World PriceRunner finds the best deals. Guess which price comparison site I am working for :-)
2005-09-22T23:23:00.000Z

David Heinemeier on Rails Scalability and the Importance of Programmer Culture

A .Net programmer asked on the Rails mailing list if the switch to Rails had really been worth it for him and whether Rails scales. David Heinemeier gives a nice and balanced reply pointing to the scalability of BaseCamphq.com and 43things.com but also stressing the importance of being in a programmer culture that you like.

By the way, I am very impressed by the Gmane mailing list web site that provides a very usable threaded web interface to mailing lists that is searchable. Kudos to the Gmane people (apparently in Norway) for volunteering to offer such an excellent service to the programming community! I feel like donating some money to them.

2005-07-30T00:10:00.000Z

Book Tip of the Day

Think and Grow Rich explains that we truly are the "masters of our fate and the captains of our souls.", or in other words:

"How to Recognize, Relate, Assimilate and Apply principles whereby you can achieve any goal whatsoever that doesn't violate Universal Law - the Law of God and the rights of your fellowman..."

2005-07-07T11:34:00.000Z

Beautiful Field in Hjørring - North Jutland - Denmark

Beautiful Field in Hjorring
2005-05-29T06:34:23.000Z

Dagens vits

Direktörerna för ölmärkena Heineken, Budweiser, Carlsberg och Guinness är ute en eftermiddag och spelar golf. Efter 18 hål på Gleneagles samlas man vid hål 19 (baren i klubbhuset). Man hann knappt sätta sig förrän en snygg servitris uppenbarade sig, och frågade vad de ville dricka.

Carlsbergdirektören skulle ha "Carlsberg. Probably the best beer in the world". Heinekenkillen fnös och beställde det han ansåg vara världens bästa öl, nämligen Heineken. Killen från Budweiser smålog och sa på släpig Texasaccent "You guy's don't know shit. Give me the best selling beer in the United States, a Budweiser". Direktören för Guinness meddelade frankt att "I'll have a Coke". Chockade frågade dom andra varför han inte beställt en Guinness, varpå Guinnessdirektören snabbt svarade "Well, if you guys don't drink beer, neither will I".

2005-04-03T21:06:51.000Z

Quote of the Day

You must give some time to your fellow men. Even if it's a little thing, do something for others - something for which you get no pay but the privilege of doing it.
-- Albert Schweitzer

2005-04-01T22:38:40.000Z

Bästa Aprilskämtet

SvD skriver att:

Dagens Nyheter avslöjar regeringens planer på att höja energiskatten i Skåne för att jämna ut orättvisan att de kan resa till Danmark och handla billigare alkohol. Nyktra skåningar ska kunna lämna in en intygan om att de är nykterister, men om de då påkomms med alkohol i kroppen får de böter för el-onykterhet eller i grova fall el-fylleri.

2005-03-29T06:35:33.000Z

Quote of the Day

Life ought to be a struggle of desire toward adventures whose nobility will fertilize the soul.
-- Rebecca West
2005-03-27T02:26:00.000Z

Voila - My new Homepage

As a result of my move to Stockholm and no longer working for Collaboraid I have now moved my homepage over to a new server at Dreamhost - an excellent LAMP hosting company in the US.

For my weblog I chose to use Wordpress and I have certainly not been disappointed. In fact, Worpress continues to impress me in so many different ways. The UI is very slick and the software feature rich. It has a great community with many themes and plugins, and it has seemed robust so far.

Most importantly, I was totally blown away by the simplicity and elegance with which I could extend Wordpress using a simple callbacks / hooks mechanism. I was able to quickly add a language filter for posts allowing users to view only posts in english, and I managed to do this without modifying the Wordpress source code!

I've only looked at the Wordpress codebase briefly, but it seems to me they are following conventions whenever they can, and keeping things simple. It's the beauty of the PHP software I've seen so far - it's very easy to find your way around the software and make small hacks and get immediate feedback on your changes.

Wordpress retains most of the typical PHP simplicity and straightforwardness and combines it with good design, very little duplication, and most notably a brilliant callback extension mechanism that preserves upgradability. I hope this kind of callback approach is more widely adopted in the industry. I think it is the way forward and finally I am hopeful that we can solve the annoying problem that has been with us for so long of allowing open source customizations and upgrades to play well together.

2005-03-27T01:32:00.000Z

Sideways

After having seen the good reviews and knowing that Sideways is the type of movie I like the most - realistic dramas with good actors and interesting dialogue - I entered the cinema with high expectations. To my surprise Sideways was even better than I thought it would be.

The movie is well made overall with nice flow and good acting. It touches on serious relationship and career issues and provides good topics for discussion. Most importantly though, Sideways feels very unpretentious and manages to be light and humorous among all the misery. It ends up being a very successful feel good movie, but a feel good movie with some weight behind it.

The dynamics and contrast between the two main male characters - the grumpy and anti-social writer and the playboy type actor - is really what makes the movie sparkle for me. There is a lot of good comedy and even a scene with some touching romance - a scene involving - of course - some fine red wine.

As a bonus the movie is set in very pleasant Californian wine yards and scenery. Much of which brought back nostalgic memories for me from my own wine tasting round trip in Sonoma Valley back in the year 2000. It's time to go back to California...

Highly recommended.

2005-01-20T07:00:00.000Z

Billiga mobilabonnemang

Jag var precis på väg att teckna DJuice kontantkort men sen såg jag domen i marknadsdomstolen om att deras prisjämförelser inte inkluderat Comviq, så nu prövar jag Comviqs kontantkort istället.
2005-01-19T07:00:00.000Z

Vänsterpartiet överrepresenterat bland journalister

Det är ganska häpnadsväckande hur stor andel av journalister som sympatiserar med vänsterpartiet. Det är ca en tredjedel och trenden är tilltagande, se denna rapport från Göteborgs Universitet.
2005-01-14T07:00:00.000Z

Dagens Ordspråk

Fråga inte efter vad världen behöver.
Fråga istället efter vad som får ditt hjärta att sjunga.
Ty vad världen behöver är människor med hjärtan som sjunger.

2005-01-14T07:00:00.000Z

Nyhetsrubriker

Det är intressant att jämföra rubrikerna som uppstått när Marita Ulvskog kritiserade kungens uttalanden om Asien katastrofen. DNs rubrik är "Ulvskog till attack mot kungen" medans SvD använder den betydligt korrektare och mer nyanserade rubriken "Ulvskog kritiserar kungligt inlägg". DNs rubrik är närmast på nivå med kvällstidningspressen. Efter att jag skrivit detta kollade jag aftonbladets första sida och kunde konstatera att deras rubrik är identisk med DNs...

Bortsett från rubriken har jag svårt att förstå att Ulvskogs uttalande är dagens huvudnyhet. Förmodligten är det för att monarkins vara eller inte vara blivit ett så hett ämne på sista tiden och naturligtvis för att Asien katastrofen fullständigt dominerar nyhetsbevakningen som om ingenting annat hände i världen.

När sedan Göran Persson blandar sig i debatten om kungens uttalande får vi samma kvalitetsskillnad i rubrikerna. SvD skriver att "Persson tonar ned kritik mot Kungen" medans DN spetsar till det med "Persson distansierar sig från Ulvskog".

Jag skickade min lilla nyhetsrubriksjämförelse till DNs chefredaktör Jan Wifstrand och hans svar var:

"kanske är det bättre att Du skriver till Aftonbladet eller SvD. Vi har inget större intresse av att jämföra oss med dessa båda mycket mera partiintressestyrda publikationer. DN sätter agendan och gör det självständigt."

Det svaret ger mig ännu en anledning att välja SvD framför DN...

En annan skillnad mellan DN och SvD tycks vara att DN (återigen i likhet med kvällstidningspressen) låter händelser som Asien katastrofen dominera nyhetsförmedlingen längre tid än SvD. Idag har SvD som första nyhet domen mot den amerikanska generalen för övergrepp mot irakiska fångar. Detta tycks också vara huvudnyheten generellt i media, i alla fall om news.google.com är någon indikation på detta. DN däremot spekulerar vidare i vem som handlat för sent i samband med Asien katastrofen i en artikel med rubriken "Persson valde att hålla tyst". Själv skulle jag föredra att vänta på kommissionens slutsatser och framförallt få en anaylys av hur en ny krisorganisation ska se ut. Men visst, DNs typ av journalistik säljer säkert för man blir alltid nyfiken när man ser den typen av rubriker. Problemet är bara att vissa läsare, åtminstone jag, ledsnar till slut...

2004-11-12T07:00:00.000Z

Search is Big but Microsoft is Bigger

In Teknisk uheld i Microsofts Google-udfordrer danish ComputerWorld writes about Microsoft's new enhanced search service based on Yahoo/Inktomi technology that is to take on Google. Supposedly the new search service has cost 600 million danish kronor to develop. That is small potato for Microsoft though where the MSN division has a turnover of a mere 2 billion danish kronor compared to 211 billion for the whole company...

2004-11-04T07:00:00.000Z

Moral Values Deciding Factor

The New York Times writes:

Of the people who chose "moral values" as their top issue, 80 percent voted for Mr. Bush. (For people who chose the economy/jobs, 80 percent voted for Mr. Kerry.) Nearly one-quarter of the electorate was made up of white evangelical and born-again Christians, and they voted four to one for Mr. Bush.

2004-10-31T06:00:00.000Z

Tracing the origins of a dance

SalsaNewYork.com has an excellent collection of articles on salsa and its origins. In the text Who owns Salsa, nationality, ethnicity and clave Norman Urquía concludes:

So a simple question like "where's salsa from" leads us to questions like "what is nationality", "what is ethnicity" and "what is identity" and the idea of a music moving around the world and forming a joint pop culture. Its not a simple subject, you could do a PhD on it (as I am) and still not find a definitive answer. So now, when people ask me "where's salsa from", I say "If you've got a couple of weeks, I can start to explain, but wouldn't you rather dance instead?"

2004-10-30T05:00:00.000Z

Quote of the Day

Dancers are the athletes of God

- Albert Einstein

Fun quotes, and quotes to help the self admiration of dancers, can be found at DanceArt.com.

2004-09-23T05:00:00.000Z

A Skeptical Perspective on the Open Source CMS Market

I am helping Greenpeace build a new Open Source CMS system based on OpenACS and I stumbled across the article Open-Source CMS: Prohibitively Fractured? by Tony Byrne, founder of CMSWatch. I think the article makes some good points for example about programming language religion, technical elegance over end user features, dilusion of efforts, the risk of forking, and re-invention of the wheel over reuse of standard APIs (for example from the Apache project).

There is quite an impressive supply of sites on the web providing listings, reviews, demos, and articles about CMS systems, such as OpenSourceCMS.com, CMSMatrix, and CMSInfo

2004-09-11T05:00:00.000Z

Launching new Website

I am just about to launch a beta version of a new little web project of mine - a salsa calendar. I will test the calendar with salsa dancing and music events here in Copenhagen to start with but the idea is to cover other cities in Europe and the US as well. I'm really excited about the website and I'm hoping people will find it useful!

2004-09-08T05:00:00.000Z

Quote of the Day

IT guru Bruce Schneier interviewed at IT Conversations says:

"More people are killed every year by pigs than by sharks, which shows you how good we are at evaluating risk."

2004-08-26T05:00:00.000Z

Floating in Hosting Paradise

I just signed up with DreamHost and I must say that so far it's been an amazingly smooth and pleasant experience. Their web control panel and help systems are really powerful and usable and everything on the server has worked great so far! DreamHost really seems to have great sys admins and they've been in the business for quite a while so they've probably learned a lesson or two over the years.

I really like managed hosting, it gives you that warm and cosy feeling of beeing well taken care of and not have to worry about things like backups and server uptime and other boring infrastructure details. Of course you still need to think about backups but at least your hosting provider has a fallback for you if your own setup fails.

Here is a list of features at DreamHost that I copied from one of their informative and amusing newsletters:

* Jabber Server.
* Quicktime Streaming.
* Miva Merchant included on a pretty cheap plan.
* We keep two hourly, two daily, and two weekly FULL backups of all 
   websites on all plans for free, and you can access them in the hidden
   .snapshot directory in every directory of every user.
* All Debian.
* Lots of Domains/SubDomains/Users/Databases/Disk/Bandwidth.
* DreamHost Rewards.
* DreamHost Donations.
* We're small(ish), never merged with anybody, and not public.
* The Sexiest Newsletter Writer in the Industry, nay, the World.
2004-08-23T05:00:00.000Z

Exotic Sailboats

There is a very impressive Oyster 82 sailboat from George Town visiting down in Nyhavn right next to my work here in Copenhagen. I found some nice photos on the web displaying this magnificent boat. Here is a German article about it as well. This summer I saw a similar type of boat in Nybroviken in Stockholm visiting from the Cayman Islands. Seems those boats are always from some exotic place far away. Myself I am content just admiring them while strolling through the harbor...
2004-08-18T05:00:00.000Z

Unfairenheit 9/11

This piece by Christopher Hitchens in the Slate magazine is worth reading. Mr Hitchens has this to say about Michael Moore's journalistic qualities:

"So I know, thanks, before you tell me, that a documentary must have a "POV" or point of view and that it must also impose a narrative line. But if you leave out absolutely everything that might give your "narrative" a problem and throw in any old rubbish that might support it, and you don't even care that one bit of that rubbish flatly contradicts the next bit, and you give no chance to those who might differ, then you have betrayed your craft."

Christopher Hitchens ends his article with this warning to his readers:

"If Michael Moore had had his way, Slobodan Milosevic would still be the big man in a starved and tyrannical Serbia. Bosnia and Kosovo would have been cleansed and annexed. If Michael Moore had been listened to, Afghanistan would still be under Taliban rule, and Kuwait would have remained part of Iraq. And Iraq itself would still be the personal property of a psychopathic crime family, bargaining covertly with the slave state of North Korea for WMD. You might hope that a retrospective awareness of this kind would induce a little modesty. To the contrary, it is employed to pump air into one of the great sagging blimps of our sorry, mediocre, celeb-rotten culture. Rock the vote, indeed."

In Fifty-nine Deceits in Fahrenheit 9/11 Dave Kopel seems to punch holes in the Michael Moore movie even more elaborately and convincingly than Christopher Hitchens did.

It's hilarious that some people are now making a movie about Michael Moore's lies, presumably using the same kind of manipulative one sided journalism that Michael Moore himself uses. Oh well...

2004-08-17T05:00:00.000Z

Swedish IT Educations No Longer In Demand

According to an article in the danish ComputerWorld the number of IT students in Sweden has fallen from 22.000 in year 2000 to less than 5.000 this year... The swedish P1 radio station just reported that during the same time period the number of patent applications are down 27%. Another measure of the IT/Telecom crash is that Ericsson that used to be the biggest private employer in Sweden with 45.000 employees is reduced to 23.000 or so today.
2004-08-12T05:00:00.000Z

Amazing Photography

I check back on photo.net every now and then and this time I found a very inspiring collection of photos by a good looking young girl with user name Fili. She doesn't match my stereotypical image of a professional photographer at all :-) Anyway, I just love landscape and mood photos like these. In fact it would be nice to have some of these photos on the walls in my apartment...
2004-07-06T05:00:00.000Z

Sliding Around in the Mud...

was what I did at this years Roskilde Music Festival. I've been wanting to go for several years and this year I finally made it. The ticket is about 1100 danish kronor for thursday to sunday - four days packed with concerts with bands and artists of all kinds, most of which I have never heard of before.

I saw a concert with Joss Stone, an oustanding concert with Tim Christensen, and good concerts with Michael Franti (reggae/rock/hip hop) and Santana. Overall I had some great musical experiences and I got take in a bit of the Roskilde atmosphere although I neither got drunk with the teenagers nor stayed in a tent in the mud fields. I think for next time though, to get the most of the festival I need to camp there. Also, and not least important, the weather needs to be better :-) Practically every day for the last couple of weeks there has been a weird mix of heavy rain and intermittent sunshine - I can't remember having seen weather like this before...

2004-06-15T05:00:00.000Z

Social Networking Abounds

I knew that dating and networking sites were in vogue, but I didn't expect there to be this many!
2004-06-01T05:00:00.000Z

Software Project Methodology

I am reading "Software Project Survival Guide" by Steve McConnell which I find very useful. Since McConnell is talking about upstream and downstream activities (waterfall approach?) and emphasizing that changes are vastly costlier when they are made late on in the project, I was curious as to how compatible his approach is with Extreme Programming. The following links provided some insight:

I found this section from McConnell's speech particularly illuminating:

Design Advice -- What has changed in 10 years?

    * In 1990s, design pundits wanted to dot every i and cross every t before writing any code
    * In 2000s, say BDUF (big design up front)? YAGNI (you aren't gonna need it)
    * There are lots of valid points on the no design--all design continuum

Extremes are usually not productive

    * All design up front vs. no
    * Entirely planned vs. improvised
    * Pure iterative vs. straight sequential
    * All structure vs. all creative
    * Document everything vs. nothing

Maybe it is in the overlap of the different software project management approaches that we, using our common sense and experience, can find the method that works best for the particulars of our own software projects.

In the book "Extreme Programming Perspectives", Pascal van Cauwenberghe offers this acronym for finding the balance between up-front work and refactoring:

"Instead of using the disparaging term "big design up front" (BDUF) we should be investigating how best to determine what is "just enough design for increments" (JEDI)."

2004-05-30T05:00:00.000Z

Quote of the Day

Edie the Salsa Freak tells the story of how she was instantly determined to learn salsa when seing Joby Martinez-Vazquez dance. In an interview Joby offers this piece of life advice:

WORK LIKE YOU DON'T NEED THE MONEY, LOVE LIKE YOU'VE NEVER BEEN HURT AND DANCE LIKE NOBODY'S WATCHING.

2004-05-29T05:00:00.000Z

Opening a File

Bruce Eckel on why he loves Python:

They say you can hold seven plus or minus two pieces of information in your mind. I can't remember how to open files in Java. I've written chapters on it. I've done it a bunch of times, but it's too many steps. And when I actually analyze it, I realize these are just silly design decisions that they made. Even if they insisted on using the Decorator pattern in java.io, they should have had a convenience constructor for opening files simply. Because we open files all the time, but nobody can remember how. It is too much information to hold in your mind.

Here is how you open a file in Python: "for line in file('filename').readlines():,". I'm increasingly motivated to learn Python and also to have a closer look at Plone.

2004-05-29T05:00:00.000Z

Joel Says it Well

Joel writes:

"That a big contradiction in Denmark: it was early in adopting real women's lib and having adult members of both sexes in the workplace, but retains by law the shop schedule that only works when each household has a homemaker. The only things you can reliably buy outside of the window of M-F 10-5, Sat 11-2 are snack foods, restaurant meals, and alcohol."

I was actually annoyed by the opening hours tyranny myself recently since I know I will have a big car with storage space available to go to Ikea next sunday, only of course, Ikea being open on a sunday is unthinkable in Denmark.

2004-05-28T05:00:00.000Z

Quote of the Day

"Many people die with their music still in them. Why is this so? Too often it is because they are always getting ready to live. Before they know it, time runs out."
--Oliver Wendell Holmes Jr., Supreme Court Justice

2004-05-21T05:00:00.000Z

The Synergy of Diana Krall and Elvis Costello

I just listened a little to Diana Krall's new CD at Amazon and it sounds very promising! I'll probably buy it this afternoon. This comment in one of the reviews, despite being a little harsh and exaggerated, struck a chord with me:

"I was NOT a Krall fan. In fact, I did not like her previous collection of stale cotton candy versions of jazz standards. They were simply boring musically and vocally. Being a hard core jazz fan, I have heard much better versions of those standards. To me she was a pretty face reintroducing standards to those who don't know they are standards. This disc, however, has made me a fan."

2004-05-04T05:00:00.000Z

Embarrassingly Successful Teaching

In the NY Times article Educators Flocking, Land of Literate Children the reasons for Finlands number one ranking in a respected international literacy survey are discussed. How do finns feel about this success?

""We are a little bit embarrassed about our success," said Simo Juva, a special government adviser to the Ministry of Education, summing up the typical reaction in Finland, where boasting over accomplishments does not come easily. Perhaps next year, he said, wishfully, Finland will place second or third."

More interestingly the article goes on to deliver the explanation for the high literacy rate:

"If one trait sets Finland apart from many other countries, it is the quality and social standing of its teachers, said Barry Macgaw, the director for education at the O.E.C.D."

According to the article, teaching is actually the favorite profession for teenagers in Finland and many more apply to become teachers than are accepted.

2004-05-04T05:00:00.000Z

A Week in New York

My intense week in the big apple passed by quicker than expected. Before memory fades further I'll take this opportunity to reminisce on the experiences I had there.

Ever since I visited New York for a weekend in the summer of 2001 I had wanted to return there. The city immediately struck me as fascinating and electrifying, intense, dense, energetic, fast talking, edgy, and ultimately very exhausting for visitors. To me it is really the city of cities and with its size and multi culturalism it justifyably considers itself "The Capital of the World".

Now that the dollar was so low and spring was coming I figured that the timing was right to revisit the city and examine it more closely. I was very fortunate to be able to stay for a week with my friend Alice on the upper east side and had midtown reachable by subway in 10 minutes or so.

Since I moved to Copenhagen almost two years ago salsa dancing has become quite a big hobby for me, and since salsa to a big extent originates from New York I was of course very excited about the opportunity to try out the dancing scene there. To guide me I found what is easily the best web site about salsa that I have found to date, namely the SalsaNewYork.com website operated by Steve Shaw. The site has great articles about salsa dancing and music and a calendar with all major New York style (aka mambo, or on 2 salsa) dance events in New York. As I arrived in New York on sunday afternoon I went more or less straight to the Jimmy Anton social dance. The place where the dance is held is in and of itself very low key and uninspiring, it's basically just a big dance studio. But when the huge dance floor filled up with loads and loads of good mambo dancers and excellent music was streaming from a good sound system the atmosphere rose to very high levels. That afternoon was my best dance experience in New York, and it makes perfect sense to me that the Jimmy Anton dance event is the favorite of SalsaNewYork.com as well. On tuesday I visited the salsa dance at the Coda club run by Tony Luna. The place was posh with a good atmosphere and a mixed crowd, only a minority of which seemed to be practiced salsa dancers though. I still had a good time there. Finally on saturday I managed to enter the China Club after having passed through a strict metal detector control, and having circumvented the $25 charge by printing a flyer from their homepage. I know $25 seems like a fortune but apparently by New York standards its a normal cover charge. The China Club has dancing on three floors: Hip Hop downstairs, techno/house on the middle floor, and salsa upstairs in the poshest part with the terrace next to the dance floor where you can go out to cool down and get a view of the night sky and the surrounding sky scrapers - quite impressive. The only thing I regret is not having visited the supposedly biggest salsa club in New York - the Copacabana - especially on the night when El Gran Combo (one of the most famous salsa bands) was playing there. I'll leave that for my next visit.

I listened to a superb latin/afro jazz concert at the Blue Note jazz club. I was sitting right next to the stage in this splendid candle lit jazz club with great ambience and service. The musicians (Mike Mossman, Conrad Herwig, Bobby Sanabria, Edsel Gomez, Mario Rivera, Ruben Rodriguez, Pedro Martinez, and Xiomara Luagart) were all oustanding and the sound was great. I enjoyed the concert so much!

I had an amazing vegetarian dinner at the Red Bamboo Vegetarian Soul Café. I never realized vegetarian meat dishes (like chicken wings) could look and taste so authentic. We got huge portions at very reasonable prices and they tasted great. Highly recommended!

I went thrift and vintage shopping with Alice and bought a whole pile of sweaters and shirts of expensive brands at bargain prices. The shirts were like $5 each and would probably be at least $50 to buy in a normal store. I searched and eventually found a shop specialized in dancing clothes on Broadway where I got the UFO pants that I had been thinking of getting for a long time (at a much lower price than in Europe of course). I spent a lot of time exploring Manhattan on foot in the sun and never tired of photographing dazzling sky scrapers as I came around every corner. I had a nice dinner with fellow OpenACS programmer Vinod Kurup at a cuban restaurant and ate hamburgers at the Blue 9 Burger that were quite tasty. According to a review at citysearch the Corner Bistro is better so I guess I'll have to try that next time :-) I wonder though if it compete with Mr. Bartley's Burger Cottage over in Boston (a place I visited a lot while I was working over there). I tried Chocolate Fudge Brownie and other wonderful tastes of ice cream at the Ben and Jerry scoop shops, had a Krispy Kreme doughnut, got in the habit of getting coffee and Bagel to go for breakfast. All in all not a very healthy food record :-)

One of the things I like the most about the US is how open, talkative, and positive people are. For me coming from a rather dull and reserved scandinavian social climate it's a quite refreshing change to be given such a friendly treatment by strangers. I of course also appreciate the nice service you get in restaurants and shops, people will really go out of their way to help you. When it comes to shopping the lower prices and greater selection is also appreciated of course. Also, I just love the Barnes and Noble type bookstores that they have in the US with their amazing selection of books at reasonable prizes combined with a cafe where you can read the books at your leisure.

2004-04-29T05:00:00.000Z

IKEA Spreading like a Virus in the US

Read about it in the Onion.
2004-04-17T05:00:00.000Z

Photos from New York

I spent an amazing week in New York visiting my good friend Alice, and I really had a great time! I took quite a few photos. On my way home from New York I spent a day in London and once again captured a couple of famous and not so famous attractions on camera (excuse the dull London weather).

New York Photo New York Photo New York Photo

2004-04-17T05:00:00.000Z

PHP - Ugly but Simple

It bodes well for the future of PHP when Rasmus Lerdorf writes that:

Despite what the future may hold for PHP, one thing will remain constant. We will continue to fight the complexity to which so many people seem to be addicted. The most complex solution is rarely the right one. Our single-minded direct approach to solving the Web problem is what has set PHP apart from the start, and while other solutions around us seem to get bigger and more complex, we are striving to simplify and streamline PHP and its approach to solving the Web problem.

2004-04-16T05:00:00.000Z

The Onion

While I was over in New York on holiday (I'll post photos soon) I discovered the Onion - a very amusing newspaper! I had heard about it before but never took the time to check it out. I find their humour quite intelligent and subtle, with a lot of irony. This article is a good example.
2004-04-01T06:00:00.000Z

The History and Rhythm of Salsa

This article by Chris Washburne is the best description that I have found so far of the origins of salsa and it also contains an alysis of the clave rhythms and how they relate to african rhythms. The article was linked off of Steve Shaws excellent SalsaNewYork.com site.
2004-03-31T06:00:00.000Z

Overcoming Some Fears Of Social Dancing

I found a good article by Steve Shaw at the info packad SalsaNewYork website and I particularly liked this passage:

Regarding the worry that some of these very good dancers are watching you and are judging and critical, I have bad news and I have good news: 1) The bad news is that unless you're a really super dancer, they're not watching you. You're being ignored. 2) The good news is that unless you're a really super dancer, they're not watching you. You're being ignored. 3) And if you are a superdancer, and they're watching you with a frown on their faces, it's probably because they envy you, and also they're hard at work trying to steal your material.

2004-03-31T06:00:00.000Z

Europe Catching up with the US on Lowprice Airlines

The Economist writes about the Airline crisis and compares the European lowprice market to the one in the US:

The European market is going the same way, and it is all happening much faster, since deregulation only arrived fully in 1997. Budget carriers such as easyJet, Ryanair and over a dozen smaller start-ups still account for less than a fifth of European air travel, but most analysts expect their market share to grow rapidly. This year the budget airlines will carry more than 50m passengers around Europe. In the past couple of months another six budget airlines got off the ground in Britain, Ireland, Hungary, Germany and Poland.

2004-03-30T06:00:00.000Z

Failing to Understand your Scandinavian Neighbours?

I've discussed it with people many times and now I finally found an investigation that shows that norwegians are best at understanding the languages of their scandinavian neighbours. In addition to that norwegian is also the language that danes and swedes best understand, swedes prefering modern norwegian (nynorsk) over the old variant (bokmaal) since the latter one has more in common with danish.

Myself I'm still struggling with danish and I've decided to take a course now to speed up my learning...

2004-03-30T06:00:00.000Z

Smoking remains constant

I was reading about the smoking ban in Ireland and found this terrible piece of statistics:

"Germany, which has one of the highest rates of smokers along with Spain and Greece, has been criticized for not doing enough to combat nicotine addiction. EU Health Commissioner David Byrnes says the government is too easy-going, considering that an average of 100,000 Germans die of smoking-related diseases each year, and that the number of smokers remains constant, with more and more young people picking up the habit."

2004-02-29T07:00:00.000Z

The origins of salsa

I was reading Amazon reviews of a Spanish Harlem Orchestra CD when I found a comment that helped clarify the much disputed and confusing origins of salsa music and dancing:

Salsa music includes all Cuban rythms such as Son, Cha Cha Cha, Mambo, Timba, Songo, Guaguanco, Rumba, Guajira and so on. All those rythms were born in Cuba and not in New York. The word Salsa was originated in New York to better market all these rythms. Please, make your own research and I suggest you start with the following movie: "Del Son a la Salsa". This movie features all time Puerto Rican greats Gilberto Santa Rosa, Marc Anthony, Cheo Feliciano, Tito Puentes, Venezuela great Oscar D' Leon, Panama master Ruben Blades and many Cuban great musicians from inside the island and from outside. All of them confirmed that Cuba is the motherland of Salsa.

2004-02-25T07:00:00.000Z

Learning Salsa

I found an interesting interview with Johnny Vazquez - one of the world's best salsa dancers who I had the honor of taking a workshop with at the salsa congress last weekend in Zurich. Here is what he says about how he learned salsa:

"I learned how to dance salsa really fast because three months after I arrived in LA, I was in a nightclub with my brother Louis, and this girl came and she told my brother, "Is that your little brother?" "Yes he is." "Oh, I'm gonna dance with him, he must be as good as you." I didn't know how to dance then, only the basics, back and forward. I said, "No I don't know how to dance." "No, no come on." She took me to the dance floor. I told her a thousand times, "I don't know how to dance. " "No, you are one of the Vazquez brothers, how can you say that?" We started dancing. After two minutes she left me on the dance floor. And she stopped and she told me, "That's all you can do, the basics?" "I told you I don't know how to dance." She told me, "Oh, so when you do learn more, then ask me to dance." And she left me. You don't know how much it hurt me. That night I was crying in my bed. Because everybody was looking at me when she left me on the floor. And being a brother of somebody who was really popular in LA, my brother Louis, it was so embarrassing for me because I didn't know how to dance. So that night I said to myself, I'm going to learn how to dance. Believe me. One year later I did the competition and when I won, she came, the same girl came to me. "Oh, I can't believe it, Johnny!" And she hugged me. "Oh, I can't believe it, when you are done with all your friends, will you ask me to dance?" And I looked at her and I said, "When you learn how to dance better, you can ask me to dance" and I turned around and I left. Now I cannot thank that girl enough because she gave me the impulse to learn."

2004-02-12T07:00:00.000Z

Coordination Challenge

While sitting at your desk, lift your right foot off the floor and make
clockwise circles. Now, while doing this, draw the number "6" in the air
with your right hand.


Your foot will change direction and there's nothing you can do about it.
2004-01-31T07:00:00.000Z

Software Methodologies - Agile RUP?

I came across the Agile Alliance homepage and took the time out to ponder over the Agile Manifesto again:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

Searching for RUP on the Agile Alliance website I found a number of interesting articles. People tend to think of RUP and XP as standing on opposite sides of the software management spectrum. There is of course some truth to that, but the Rationial whitepaper below argues that RUP is "a processes framework", and that an implementation of RUP to small projects with a small number of artifacts can become a close fit to XP. The paper further argues that XP is vulnerable to staff turnover (process experience is not documented and preserved).

» Next page