Peter Marklund

Peter Marklund's Home

Peter on Rails

Lessons Learned in Ruby on Rails and related technologies. Subscribe

Sat December 17, 2005
Programming

How I Test my Website

Introduction

This is an article that talks about the motivation for testing and how I went about writing automated tests for my personal website. It is best suited for someone with at least some familiarity with Rails but who hasn't yet explored its testing capabilities. There is also a section on web testing that is independent of Rails.

I learned my lesson in Berlin

I still remember clearly the somewhat embarrassing mistake that sparked my interest in software testing. I was working as an intern for a research institute in Berlin and my task was to program an image processing algorithm in C. Once the program was finished I was stalled for a long time, maybe weeks, puzzled over the unexpected outputs of my program. My supervisor was starting to lose patience with my lack of progress. Eventually it occurred to me that the underlying algorithm that I had implemented was broken due to a trivial mistake. All along I had simply assumed that it was working without conducting any thorough tests to prove my assumption. That's when it dawned on me what a challenging and important problem software testing is.

The miserable past of web testing

A lot of developers see testing as a dull or even unnecessary activity. The level of testing has been particularly poor in web development, a sector of the software industry notorious for being hackish and low in quality. Granted, the cost of a broken web page is typically not as big as a bug in banking software. However, a broken page on a high volume public site or an internet banking site can be very costly indeed.

With Agile methodologies such as Extreme Programming the idea is to make changes, refactor, and release often. Good automated test coverage is almost a prerequisite for this style of development. This is because for each change that we make we need to make sure that we didn't break existing functionality. In a system with high coupling the bug caused by a change can arise in a seemingly unrelated part of the system. Of course, manual testing is not really an option in agile development as it would be too expensive and slow down the development and release cycle too much.

There are benefits to testing other than just finding bugs that are worth mentioning. One is that testing forces you to focus on and define the interface (aka contract) of your application or API. This is the reason why test driven development can be so valuable since defining the contract of your code early on provides a good direction and goal for your programming - it serves as a specification. Also, when you are writing tests you will often find yourself reviewing your code and this review in and of itself can reveal valuable improvements.

Rails shows the way

Most of the hype surrounding Rails has to do with its productivity and the surprisingly few lines of code needed to build a typical website. A nice side effect of having a small code base is that you have fewer lines of code to debug. Testing in Rails is also helped by its strict MVC (Model-View-Controller) structure. Last but not least, Rails comes shipped with a tightly integrated testing framework that allows testing both on the controller and model levels.

Rails has an interesting code statistics tool invoked by "rake stats" that counts the lines of code (LOC) and the ratio of production to test code. Here is the output for my website:

+----------------------+-------+-------+---------+---------+-----+-------+
| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Helpers              |    19 |    18 |       0 |       1 |   0 |    16 |
| Controllers          |   260 |   210 |       6 |      37 |   6 |     3 |
| Components           |     0 |     0 |       0 |       0 |   0 |     0 |
|   Functional tests   |   340 |   253 |      10 |      45 |   4 |     3 |
| Models               |   122 |    91 |       4 |      12 |   3 |     5 |
|   Unit tests         |   191 |   150 |       3 |      17 |   5 |     6 |
| Libraries            |     0 |     0 |       0 |       0 |   0 |     0 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total                |   932 |   722 |      23 |     112 |   4 |     4 |
+----------------------+-------+-------+---------+---------+-----+-------+
  Code LOC: 319     Test LOC: 403     Code to Test Ratio: 1:1.3

Unit testing

A unit test in Rails tests a single model class. For my website I have three model classes - Post, Category, and User. There is a one-to-one mapping in Rails between model classes and database tables. The model classes have methods for the usual CRUD (create, read, update, delete) operations and all of these operations interact with the database.

When you write tests that depend on database data it will simplify your life a lot if the tests have control over that data. This is one of the areas where the Rails testing framework shines. All Rails tests are executed against a separate test database into which data is loaded with the help of fixtures. Each database table has a corresponding fixture YAML file. YAML is a concise and yet readable text data format that is quite handy to work with. The YAML files can contain embedded ruby code and so can be dynamic with loops and variables.

Here are some examples of what my unit tests cover:

Functional testing

In Rails functional tests are applied at the controller level. Each controller class has a corresponding controller test class. The controller test executes actions on its controller, makes assertions about the values of the member variables in the controller, about the type of response (success, redirect etc.), and about the HTML output generated by the view. The controller test also has access to the request and response objects, to the session, and to the flash object used to display status messages to the user. All in all this makes controller tests very powerful.

Here are some functional tests I wrote for my website:

One minor limitation of controller tests to be aware of is that they don't include the URL-action mappings (the routes). One could probably test those separately somehow but I haven't seen an example of that yet. The URL mappings are covered by the web tests though, see below.

There is potential for overlap between unit and functional testing. Thorough testing at the model level decreases (but does not eliminate) the need for testing at the controller level. It is clearly an advantage if the controller tests can rely on the underlying model to be working correctly so that they can focus on the controller specific logic.

Web testing and Monitoring

I use web testing for pre-launch and post-launch acceptance testing and for monitoring of my website. This means I run the web tests just before a site upgrade, immediately after the upgrade, and then regularly every day against the production site. The web tests execute the entire system including the production database. The fact that the production database is used is important since it is not uncommon for bugs to depend on database data. The Rails unit and functional tests can provide good test coverage but the web tests are a necessary complement and are the ultimate check that the system is working at the point of release and that it keeps working in production.

I have divided my web tests into two parts. I have one "ping" test that I run frequently and that just makes sure that the front page works and the site is up. The other test is a smoke test that I run twice a day. For all pages on my site it checks that:

If any of the web tests fail I am alerted by email. I am also alerted by email if an error occurs in the code (i.e. an exception is thrown). With this setup I can rest assured that if major functionality on the site breaks down I will know about it and have a good chance of taking action quickly.

Web testing is not provided by the Rails framework and I have chosen to rely on Perl scripts for those tests. My web tests currently don't do much HTML parsing, but if there was a need for that Perl has its simple yet powerful look_down method. However, if there were similar APIs available in Ruby I would probably switch to Ruby as I am more comfortable with that language.

What's Missing?

There should be some kind of automatic test that verifies that site backups are working. A way to do that would be to have a mirror site in a different location that is updated nightly with the database backup from the production site. One could then run the web tests against the mirror site as well. One would also need to verify, through a web test or otherwise, that no data is missing in the backup. If the production site is not changed during the recreation of the mirror site a sufficient test would be to check if the two sites are identical.

If a website makes heavy use of JavaScript and Ajax then an in-browser testing tool such as Selenium may be useful for acceptance testing and browser compatibility testing.

Last, but not least, if you are expecting a lot of traffic you should look at load testing with tools such as Apache Bench (ab), or siege. Basic usage of these tools is shown in the Rails book. Rails also has the concept of performance tests that make it easy to load huge quantities of data and then repeatedly execute some critical action and measure its average execution time.

Conclusion

The level of testing that I have described here isn't comprehensive and really ought to serve as a baseline for pretty much any serious website. Unfortunately, this is far from the case in the industry today. With the advent of Rails, testing is more readily available to web developers than ever before. Let's remind ourselves to take full advantage of that.

1 Comment

Tue December 06, 2005
Programming

The Threat of New Technologies

In the midst of one of the many Java/Rails flamewars (haven't we had enough of them already?) I found a thought provoking comment by the creator of Tapestry on Rails and the resistance to technology change among developers:

"It's natural to be a little scared of new developments. It's painful to think that your exclusive, hard-won skills may be even a little bit obsolete. When I see Flex, Laszlo or Rails, my stomach does a little flip-flop in terms of all the effort I've put into Tapestry. But the reality is that *all* of these approaches are transitory (I'm sure we'll all be doing something quite different in five years), and we should embrace change, learn from other's efforts, and create even better frameworks and applications."

"For example, there may come a time when much Tapestry work is done using Trails (a Tapestry/Rails/Spring/Hibernate hybrid). Trails may never have offer quite the same developer productivity as Rails, but it is still a gateway into full-blown Tapestry and J2EE development."

Openness to new programming languages and toolkits seems to be a common characteristic among leaders in the programming community. The best example of this is probably the Pragmatic Programmers who came up with the now famous advice to learn one new programming language a year. I can't help but wonder that maybe if Dave Thomas hadn't picked up and documented Ruby, David Heinemeier Hansen would not have discovered it either and Rails would not have been a reality today.

I have been impressed that so many Java heavy weights (David Geary/JSF, James Duncan Davidson/Ant, Bruce Tate etc.) have had the courage to step up and recommend Rails. Personally I'm certainly ready and well equipped to make the transition to Rails and the urge to do so is bigger than ever now that I am back working in the less productive and overly complicated Java world.

Make a Comment

Sun November 27, 2005
Programming

I'm on Rails Now

I was finally able to put my new homepage live late last night after too many hours of programming (why do things always take longer than we expect?). About two weeks ago I caught a cold and I spent several days alone in my apartment. The upside of being sick is that I've had time to work very intensely on my homepage (when I probably should have been resting)...

This is the third incarnation of my homepage. A couple of years ago I started out with an OpenACS site using Lars Pind's weblog software. Since OpenACS hosting is hard to come by and that technology had little momentum I moved to dreamhost.com and Wordpress which both have worked out very, very nicely for me. At that point I didn't know the Rails revolution was coming...

When I was working for Collaboraid with Lars last year we had David Heinemeier Hansson (the creator of Rails) come visit us in our office and tell us about his toolkit. I remember how we were shooting all kinds of tough questions at him of the type "ok, that sounds good, but can Rails do this or that particular thing that we need". David exuded confidence and would patiently and immediately answer each and every one of our questions like "sure you can do that, or in fact, you can do something even better, let me show you...". We were all very convinced. At that time though, I didn't have the time or energy to pick up a new web technology and I was somewhat deterred by the fact that Rails was still such a small niche. Having lived in the OpenACS niche for so long I was very impatient to find my way back into the mainstream.

Rails stayed in the back of my mind though and when I came to Stockholm to work as a Java developer at Pricerunner I noticed how people were talking about Rails. I then realized just how big the technology must have gotten. Over the past months I've read countless articles about Rails and it's really mind boggling to see how Rails had litterally taken the world by storm. I didn't see that coming, at least not at the rate that it's been happening. There is a myriad of open source web toolkits out there competing for attention and people like me who have worked with OpenACS know how hard it is to get a toolkit using an unpopular programming language into the mainstream (there are other more compelling reasons why OpenACS never entered the mainstream, but that's a different story).

When I had realized that the Rails way is really the future of web development, and also had experienced first hand how wonderful the Ruby language is, I knew I had to learn these technologies. I figured that what better way to do that than to build my own homepage using them? I started by trying Typo - the Rails weblogging software that all the other cool Rails hackers seem to be using. However, at the time that I tested it I was kind of put off by some bugs that it had. It seemed awfully immature coming from Wordpress. Also, Rails is not a framework for plugging in and reusing applications (like OpenACS is). Rather, it is a framework for building applications from scratch in a very fast and elegant fashion.

Of course, building any application, even if it's something seemingly simple as a weblog, is going to take a lot of work and thought if you want the result to be top class. What's fascinating is that Rails makes it feasible (because of its productivity) to build from scratch in a lot of cases where previously application reuse was the only realistic option.

I'm really thrilled about web development again know that I know Rails. I'll write up some details about how I built my homepage later on.

My old homepage is still accessible and now lives at old.marklunds.com.

1 Comment

Fri November 25, 2005
Programming

Java is not Always the Best Tool? Really?

I really like the article Ruby the Rival over at OnLamp.com, and especially this part by James Duncan Davidson (this has been said many times before - when all you have is a hammer everything looks like a nail):

"And I think that's the real win from the recent attention on Ruby on Rails and the breakaway from viewing the world with Java-colored glasses. It's not that Ruby on Rails is going to be the next Java. Far from it. It's that Ruby on Rails helps to break this idea that there is "One True Way." There's not. There are many different ways to solve a problem. And really, none of them is the clear-cut winner. There's just places where one solution has advantages."

Make a Comment

Mon October 31, 2005

PriceRunner is Number One

According to PC World PriceRunner finds the best deals. Guess which price comparison site I am working for :-)

Make a Comment

Fri September 23, 2005
Programming

David Heinemeier on Rails Scalability and the Importance of Programmer Culture

A .Net programmer asked on the Rails mailing list if the switch to Rails had really been worth it for him and whether Rails scales. David Heinemeier gives a nice and balanced reply pointing to the scalability of BaseCamphq.com and 43things.com but also stressing the importance of being in a programmer culture that you like.

By the way, I am very impressed by the Gmane mailing list web site that provides a very usable threaded web interface to mailing lists that is searchable. Kudos to the Gmane people (apparently in Norway) for volunteering to offer such an excellent service to the programming community! I feel like donating some money to them.

Make a Comment

Sat July 30, 2005

Book Tip of the Day

Think and Grow Rich explains that we truly are the "masters of our fate and the captains of our souls.", or in other words:

"How to Recognize, Relate, Assimilate and Apply principles whereby you can achieve any goal whatsoever that doesn't violate Universal Law - the Law of God and the rights of your fellowman..."

Make a Comment

Thu July 07, 2005

Beautiful Field in Hjørring - North Jutland - Denmark

Beautiful Field in Hjorring

Make a Comment

Sun May 29, 2005
Humor

[se] Dagens vits

Direktörerna för ölmärkena Heineken, Budweiser, Carlsberg och Guinness är ute en eftermiddag och spelar golf. Efter 18 hål på Gleneagles samlas man vid hål 19 (baren i klubbhuset). Man hann knappt sätta sig förrän en snygg servitris uppenbarade sig, och frågade vad de ville dricka.

Carlsbergdirektören skulle ha "Carlsberg. Probably the best beer in the world". Heinekenkillen fnös och beställde det han ansåg vara världens bästa öl, nämligen Heineken. Killen från Budweiser smålog och sa på släpig Texasaccent "You guy's don't know shit. Give me the best selling beer in the United States, a Budweiser". Direktören för Guinness meddelade frankt att "I'll have a Coke". Chockade frågade dom andra varför han inte beställt en Guinness, varpå Guinnessdirektören snabbt svarade "Well, if you guys don't drink beer, neither will I".

2 Comments

Sun April 03, 2005

Quote of the Day

You must give some time to your fellow men. Even if it's a little thing, do something for others - something for which you get no pay but the privilege of doing it.
-- Albert Schweitzer

Make a Comment

Sat April 02, 2005
Humor

[se] Bästa Aprilskämtet

SvD skriver att:

Dagens Nyheter avslöjar regeringens planer på att höja energiskatten i Skåne för att jämna ut orättvisan att de kan resa till Danmark och handla billigare alkohol. Nyktra skåningar ska kunna lämna in en intygan om att de är nykterister, men om de då påkomms med alkohol i kroppen får de böter för el-onykterhet eller i grova fall el-fylleri.

Make a Comment

Tue March 29, 2005

Quote of the Day

Life ought to be a struggle of desire toward adventures whose nobility will fertilize the soul.
-- Rebecca West

Make a Comment

Sun March 27, 2005
Programming

Voila - My new Homepage

As a result of my move to Stockholm and no longer working for Collaboraid I have now moved my homepage over to a new server at Dreamhost - an excellent LAMP hosting company in the US.

For my weblog I chose to use Wordpress and I have certainly not been disappointed. In fact, Worpress continues to impress me in so many different ways. The UI is very slick and the software feature rich. It has a great community with many themes and plugins, and it has seemed robust so far.

Most importantly, I was totally blown away by the simplicity and elegance with which I could extend Wordpress using a simple callbacks / hooks mechanism. I was able to quickly add a language filter for posts allowing users to view only posts in english, and I managed to do this without modifying the Wordpress source code!

I've only looked at the Wordpress codebase briefly, but it seems to me they are following conventions whenever they can, and keeping things simple. It's the beauty of the PHP software I've seen so far - it's very easy to find your way around the software and make small hacks and get immediate feedback on your changes.

Wordpress retains most of the typical PHP simplicity and straightforwardness and combines it with good design, very little duplication, and most notably a brilliant callback extension mechanism that preserves upgradability. I hope this kind of callback approach is more widely adopted in the industry. I think it is the way forward and finally I am hopeful that we can solve the annoying problem that has been with us for so long of allowing open source customizations and upgrades to play well together.

Make a Comment

Sun March 27, 2005
Movie Reviews

Sideways

After having seen the good reviews and knowing that Sideways is the type of movie I like the most - realistic dramas with good actors and interesting dialogue - I entered the cinema with high expectations. To my surprise Sideways was even better than I thought it would be.

The movie is well made overall with nice flow and good acting. It touches on serious relationship and career issues and provides good topics for discussion. Most importantly though, Sideways feels very unpretentious and manages to be light and humorous among all the misery. It ends up being a very successful feel good movie, but a feel good movie with some weight behind it.

The dynamics and contrast between the two main male characters - the grumpy and anti-social writer and the playboy type actor - is really what makes the movie sparkle for me. There is a lot of good comedy and even a scene with some touching romance - a scene involving - of course - some fine red wine.

As a bonus the movie is set in very pleasant Californian wine yards and scenery. Much of which brought back nostalgic memories for me from my own wine tasting round trip in Sonoma Valley back in the year 2000. It's time to go back to California...

Highly recommended.

Make a Comment

Thu January 20, 2005

[se] Billiga mobilabonnemang

Jag var precis på väg att teckna DJuice kontantkort men sen såg jag domen i marknadsdomstolen om att deras prisjämförelser inte inkluderat Comviq, så nu prövar jag Comviqs kontantkort istället.

Make a Comment

Wed January 19, 2005
Politics

[se] Vänsterpartiet överrepresenterat bland journalister

Det är ganska häpnadsväckande hur stor andel av journalister som sympatiserar med vänsterpartiet. Det är ca en tredjedel och trenden är tilltagande, se denna rapport från Göteborgs Universitet.

Make a Comment

Fri January 14, 2005
Quotes

[se] Dagens Ordspråk

Fråga inte efter vad världen behöver.
Fråga istället efter vad som får ditt hjärta att sjunga.
Ty vad världen behöver är människor med hjärtan som sjunger.

1 Comment

Fri January 14, 2005
Politics

[se] Nyhetsrubriker

Det är intressant att jämföra rubrikerna som uppstått när Marita Ulvskog kritiserade kungens uttalanden om Asien katastrofen. DNs rubrik är "Ulvskog till attack mot kungen" medans SvD använder den betydligt korrektare och mer nyanserade rubriken "Ulvskog kritiserar kungligt inlägg". DNs rubrik är närmast på nivå med kvällstidningspressen. Efter att jag skrivit detta kollade jag aftonbladets första sida och kunde konstatera att deras rubrik är identisk med DNs...

Bortsett från rubriken har jag svårt att förstå att Ulvskogs uttalande är dagens huvudnyhet. Förmodligten är det för att monarkins vara eller inte vara blivit ett så hett ämne på sista tiden och naturligtvis för att Asien katastrofen fullständigt dominerar nyhetsbevakningen som om ingenting annat hände i världen.

När sedan Göran Persson blandar sig i debatten om kungens uttalande får vi samma kvalitetsskillnad i rubrikerna. SvD skriver att "Persson tonar ned kritik mot Kungen" medans DN spetsar till det med "Persson distansierar sig från Ulvskog".

Jag skickade min lilla nyhetsrubriksjämförelse till DNs chefredaktör Jan Wifstrand och hans svar var:

"kanske är det bättre att Du skriver till Aftonbladet eller SvD. Vi har inget större intresse av att jämföra oss med dessa båda mycket mera partiintressestyrda publikationer. DN sätter agendan och gör det självständigt."

Det svaret ger mig ännu en anledning att välja SvD framför DN...

En annan skillnad mellan DN och SvD tycks vara att DN (återigen i likhet med kvällstidningspressen) låter händelser som Asien katastrofen dominera nyhetsförmedlingen längre tid än SvD. Idag har SvD som första nyhet domen mot den amerikanska generalen för övergrepp mot irakiska fångar. Detta tycks också vara huvudnyheten generellt i media, i alla fall om news.google.com är någon indikation på detta. DN däremot spekulerar vidare i vem som handlat för sent i samband med Asien katastrofen i en artikel med rubriken "Persson valde att hålla tyst". Själv skulle jag föredra att vänta på kommissionens slutsatser och framförallt få en anaylys av hur en ny krisorganisation ska se ut. Men visst, DNs typ av journalistik säljer säkert för man blir alltid nyfiken när man ser den typen av rubriker. Problemet är bara att vissa läsare, åtminstone jag, ledsnar till slut...

Make a Comment