deferred until inspiration hits

Testing ActionMailer::Base.server_settings

Posted by Chris Roos Fri, 07 Dec 2007 17:23:00

I needed to use an external SMTP server to get mail through to certain providers (notably, hotmail).

I was sending my action mailer email but couldn’t be sure that I’d configured the settings correctly as I didn’t get any detailed log output (or at least not where I was looking).

I dug into the code and found the lines that were responsible for sending the email. Replaying that code in the console allowed me to identify the problem immediately (a missing domain in my case). I thought I’d paste those few lines here in case they’re of some use to someone else.

message, from, to = 'test message', 'sender-email', 'recipient-email'
s = ActionMailer::Base.server_settings
Net::SMTP.start(s[:address], s[:port], s[:domain], s[:user_name], s[:password], s[:authentication]) do |smtp| 
  smtp.sendmail(message, from, to)
end

Tags action, action-mailer, mailer, rails, ruby,

Monkey patching rails routing

Posted by Chris Roos Mon, 19 Mar 2007 22:16:00

This plugin offers the following two features.

Permanently redirecting requests for urls that end in a slash to the equivalent url with no slash, helping disambiguate urls, as explained by Simon Willison.
The addition of ActionController::Routing::RouteSet::Mapper#permanently_redirect, allowing us to declare permanent redirections within our route config.

ActionController::Routing:Routes.draw do |map|
  map.permanently_redirect 'legacy/view/:id', :controller => 'current', :action => 'show'
end

A quick google reveals that Rob McKinnon has an alternative suggestion for removing the trailing slashes.

The plugin is known to work with Rails 1.1.6.

Tags rails, routing,

Encapsulation in Active Record objects

Posted by Chris Roos Fri, 12 Jan 2007 01:37:00

I’ve been meaning to finish a post on this subject for ages. As I probably won’t finish it in its current incarnation, I thought I’d at least get something up here.

Active Record associations are brilliant for quickly putting together an object graph. They’re not so brilliant for testing, or keeping the code particularly tidy. They quickly lead to (and even encourage) train-wreck expressions in your code. Diving straight into an example:

class Person < ActiveRecord::Base
  has_many :hobbies
end
class Hobby < ActiveRecord::Base
  belongs_to :person
end

Because the hobbies object is publicly accessible from a person object, it allows us to reach in and touch things we shouldn’t.

cycling = Hobby.new

# We shouldn't manipulate the hobbies object directly...
chris = Person.new
chris.hobbies.add(cycling) # Uh oh - touching things we shouldn't

# ...rather, it'd be better to add a wrapper method for the functionality we actually need
class Person < ActiveRecord::Base
  def add_hobby(hobby)
    hobbies.add(hobby)
  end
end
chris = Person.new
chris.add_hobby(cycling) # Ahh, that's much better

The revised Person class above follows the law of demeter. As well as making the code look a little neater (no train-wrecks to see here), it makes it easier to test. To illustrate the testing point, I’m going to extend the association in this example.

# We end up with a large and not very clear test
class Person < ActiveRecord::Base
  has_many :hobbies do
    def find_favourite
      find(:first, :conditions => ['favourite = true'])
    end
  end
end

class MyContrivedTest < Test::Unit::TestCase
  def test_should_find_my_favourite_hobby
    hobbies = mock
    hobbies.expects(:find_favourite)
    person = stub(:hobbies => hobbies)
    hobby_finder = HobbyFinder.new
    hobby_finder.find_favourite(person)
  end
end

# This way, we end up with a slightly cleaner looking class (imo) and a smaller, cleaner test
class Person < ActiveRecord::Base
  has_many :hobbies
  def find_my_favourite_hobby
    hobbies.find(:first, :conditions => ['favourite = true'])
  end
end

class MyContrivedTest < Test::Unit::TestCase
  def test_should_find_my_favourite_hobby
    person = mock
    person.expects(:find_my_favourite_hobby)
    hobby_finder = HobbyFinder.new
    hobby_finder.find_favourite(person)
  end
end

Interestingly enough, if we had test driven the development of the above code, we would almost certainly have ended up with something similar (interface-wise) to the example that has a wrapper around the association proxy.

So, with all of this in mind, I’ve hacked together a real quick plugin that adds a corresponding _private method for each of the four (has_one, has_many, belongs_to and has_and_belongs_to_many) associations. Using the _private alternative will do exactly the same as the original method but it will make all of the dynamically created methods private. So. It’ll still be possible to write the wrapper around the association (as we did in the test above) but it won’t be possible to reach in and touch things that we shouldn’t.[1]

There is at least one problem (there may be many more I’m not aware of) with using the _private association methods. If you have validation set-up for one of the associations (e.g. validates_associated) it will fail. This is because validation happens from outside the instance of the object and therefore needs to access the association. Although I haven’t thought this through fully, I’m actually wondering if validation (in its current form) takes a bit of a back seat under this new world order. I think that one of the reasons validation is on the object itself is because there are many many ways to create the said object. However, using these private associations and disabling the persistence methods (create, save, update etc) on an associated object may allow us to have a single point of object creation and persistence. If we had that, then we wouldn’t need the Active Record validations anyway. Just a thought…

Just one final note. Dave Astels has a good article about encapsulation in rails, as referred to by Ben

¹ Ok. That is, of course, a blatant lie – we can still access the private association – this is ruby remember. The important point is that we’ve made our intentions more explicit.

Tags encapsulation, rails, ruby, testing,

Connecting to gmail with Ruby (or Connecting to POP3 servers over SSL with Ruby)

Posted by Chris Roos Tue, 24 Oct 2006 22:42:00

Short version (or, “I want to connect to gmail NOW”)

Download the pop3 library from Ruby HEAD (Ruby 1.9) (download file)
Use the following script to get started (making sure that you explicitly reference the downloaded pop3 file)...

require 'pop_ssl' # I renamed the file from pop.rb to pop_ssl.rb to ensure I was requiring the correct version

username = 'YOUR_GMAIL_USERNAME@gmail.com'
password = 'YOUR_GMAIL_PASSWORD'

Net::POP3.enable_ssl(OpenSSL::SSL::VERIFY_NONE)
Net::POP3.start('pop.gmail.com', 995, username, password) do |pop|
  if pop.mails.empty?
    puts 'No mail.'
  else
    pop.each_mail do |mail|
      p mail.header
    end
  end
end

Medium version (or, “Let me know I’m connecting to the correct site, Dammit”)

This is identical to the short version except that we verify the ssl certificate this time round.

Follow the first step above (i.e. download new version of pop3 lib).
Download the CA Root Certificates bundle from cURL site (download file)
Copy the code above but change the Net::POP3.enable_ssl line to read…

Net::POP3.enable_ssl(OpenSSL::SSL::VERIFY_PEER, '/PATH/TO/DOWNLOADED/cacert.pem')

Longer version (or, “Pray tell, how did you find this information sir”)

An explanation of how I got here. Feel free to ignore.

For a while I’ve wanted to play around with receiving emails within a rails app (with action mailer). I had an hour or two free the other evening and so started to play. First step was to take a look at the rails wiki and see that there’s already a page detailing exactly what I wanted to do. As I started reading, I was getting put off by the thought of configuring sendmail or some other ‘heavyweight’ solution. Luckily, someone has added a much better suggestion at the bottom of the page – using net/pop (or net/imap) to collect emails from a pre-configured mail server. Although I use imap for my own email provision, I wanted a pop server for testing. Aha, I thought, I’ll just set-up another gmail account and use that… And that’s where the problems started.

You see, gmail only accepts POP3 connections over SSL (on port 995), a feature not found in our beloved net/pop library. Hmm. Time for some help from google. The first page I came across described using stunnel to create an ssl tunnel to route standard net/pop traffic through. This sounded great, except that I couldn’t get stunnel (v4.16) to compile on mac os x, although I didn’t try too hard… The other point of interest in that original article was that the pop3 with ssl support was in development. The next step was to track down some mailing list threads that talked about the ssl support and hope for a patch. We’re in luck, the first patch was actually submitted in March 2004, with the second, accepted and committed, patch coming in April 2004. As per the last message in the second thread, the patch was accepted into core, and is now available in the 1.9 release. So, we don’t even need to patch our existing copy of net/pop, we can just download the newer version. I named this downloaded version as pop_ssl, to avoid confusion, and placed it in my ruby load path.

Next step was to follow the example on the net/pop3 page, and, with the addition of the Net::POP3::enable_ssl directive, try to connect. Hmm, things still weren’t working quite right. I was getting the following error.

OpenSSL::SSL::SSLError: certificate verify failed

Some more googling led me to this post by Greg Houston. It turns out that the error is actually raised in the OpenSSL layer. It is caused by us trying to verify the server certificate without having the correct trusted root certificate. It turns out that we can either disable the verification (by using Net::POP3::enable_ssl(OpenSSL::SSL::VERIFY_NONE)), or we can verify against a trusted root certificate bundle, like the one supplied by cURL. To verify using the certificate bundle from cURL (or similar), we would amend the above line of code to Net::POP3.enable_ssl(OpenSSL::SSL::VERIFY_PEER, File.dirname(__FILE__) + '/PATH/TO/DOWNLOADED/cacert.pem').

I can finally now get on with my rails app to retrieve emails from gmail.

Conclusion

I’ve spent a lot more than the intended couple of hours investigating this stuff, yet still have no rails app receiving emails (although I’m a lot closer now…)
There’s very little information on the ssl enabled libraries (net/https, net/pop, http-access2 to name but a few) in ruby. Maybe everybody else just knows about this stuff already, but I’m certainly pretty unclear.
I have a feeling that it should be possible to use net/https to connect (unverified) to a site that uses ssl and obtain the certificate for use in subsequent verified connections. Indeed, another document hosted on curl.haxx.se seems to indicate that this is possible.
I still know very little about ssl and certificates.
I hope that what little information is here can help someone else in a similar situation.

Tags gmail, pop, rails, ruby,

In memory AR object for testing

Posted by Chris Roos Sat, 09 Sep 2006 12:03:00

I’ve previously written about using in memory active record objects for testing.

In doing so recently, I’ve found it necessary to define the id of the in memory object. By default this is obviously nil as it has not been persisted. In addition, it is not possible to set it during normal object construction.

person = Person.new(:id => 1)
p person.id
#=> nil

However, we can give the person constructor a block, setting the id just after creation.

person = Person.new { |p| p.id = 1 }
p person.id
#=> 1

Cool.

Tags rails, testing,

Rails leaking some strings in development

Posted by Chris Roos Tue, 22 Aug 2006 10:47:00

I spent some time this morning checking out Scott Laird’s memory profiler against a fresh rails app.

I was suprised to see that around 110 strings were being leaked on each request. I was incredibly surprised when I used the string_debug option (to the memory profiler) to generate logs of strings living in ObjectSpace. There were strings appearing that I knew were not in my new rails app anywhere.

After grepping my working directory for some of these strings (and not coming up with anything), James suggested that the strings could be in my irb history. Although it wasn’t immediately obvious how they were appearing, the strings were indeed in my history (just over 100 lines worth.)

A little inspection and the culprit was found. The rails breakpoint library requires irb, which loads the irb_history.

In summary: the contents of irb_history are loaded into object space on each request made to your rails app, while in development. These strings do not appear to get garbage collected. The problem does not affect apps running in production as the breakpoint library is not loaded in that environment.

Tags memoryleak, rails, ruby,

I guess that's one way to do it...

Posted by Chris Roos Thu, 17 Aug 2006 08:08:00

We’re not quite at this stage yet…

class ActiveRecord::ConnectionAdapters::MysqlAdapter
  alias :execute_before_test_hijack :execute
  def execute(*args)
    unless args.first =~ /^SHOW FIELDS FROM/ || %w[BEGIN COMMIT ROLLBACK END].include?(args.first)
      raise "You know you're not allowed to use the database in your tests..." # We could be more gentle with a warn
    end
    execute_before_unit_test_hijack(*args)
  end
end

Tags database, rails, ruby, testing,

Tickling link_to into submission

Posted by Chris Roos Mon, 14 Aug 2006 23:10:00

What a crappy title. Ho hum.

This bit explains the itch I had. Skip over it if you like.

I recently finished re-structuring some of the code at work, in which I namespaced all of the admin type controllers. At the same time, I attempted to split some of the larger, confused (about their responsibilities) controllers into smaller chunks. This all seems to have worked quite well, but we’ve ended up with some naff (I haven’t used that word in ages) action names.

I like, where possible, to follow the scaffold naming convention (list, show, new, create, edit, update and destroy); and so went ahead and started changing things. I was writing my tests like mum always said I should¹ when I realised that there was a bit missing (uh oh). You see, I was ensuring that my controllers responded to certain actions; and I was also ensuring that my views had links to certain controllers and actions. The missing link was a check as to whether the links in my views went to valid controller actions. I could, for example, have been testing for a link to /foo/edit in my view but for an edit_foo action in my foo_controller test. Both tests would pass, but the application would still be broken. Not cool. We do actually have something in place to perform these tests but I’m still a bit scared of selenium and don’t use it as much as I should.

Anyway, i started thinking about bending the link_to helper to my will. I figured that the little fella was a good candidate to not only give me a perfectly formed chunk of html goodness; but to also tell me whether the said chunk of html goodness was actually going to work (i.e. whether my app responded to the generated url). Cool idea huh.

I’m getting tired and a little bored now so I’ll wrap this post up. I’ve created a plugin (svn co svn://dev.seagul.co.uk/plugins/action_view_helper_url_helper) that replaces the link_to method to add a class attribute (invalid_href_target or valid_href_target) to the generated anchor tag. You could then use the assert_no_broken_links assertion to ensure there are no anchors with a class of invalid_href_target in your views; and/or you could style it (see resource/style.css as an example) so that you can see any broken links at a glance whilst in development (as shown in the image below).

The code is pretty poor/naive, works with rails 1.0.0 (hence my trying to set-up a rails 1.0.0 app earlier), doesn’t work with rails 1.1.6 (not sure about versions in between) and may or may not work for you. I kinda think it’s a good idea but there’s every chance that a) it’s not or b) there’s already a simple solution to this problem.

Let me know how you get on if you decide to risk the health of your project by trying this plugin.

¹ She didn’t say that.

Tags plugin, rails, ruby,

Create rails projects using a specific rails gem version

Posted by Chris Roos Mon, 14 Aug 2006 22:06:00

I just found this when trying to create a rails project using rails 1.0.0 (having both 1.0.0 and 1.1.6 installed as gems).

If the first argument to the rails command is a valid (i.e. installed) version number and surrounded by underscores then that version of the gem is used to create the rails application structure.

So (with 1.0.0 and 1.1.6 gems installed)..

$ rails _1.0.0_ my-rails-proj
#-> uses rails version 1

$ rails my-rails-proj
#-> uses latest rails version (1.1.6)

Although you can normally place the rails libraries into the vendor directory of a given rails app (to use that specific rails version); I had problems when placing rails 1.0.0 into a rails 1.1.6 generated app. An error is encountered when trying to boot webrick..


./script/../config/../vendor/rails/railties/lib/initializer.rb:256:in `send': undefined method `cache_template_extensions=' for ActionView::Base:Class (NoMethodError)

By passing the version argument to the rails command and then placing rails 1.0.0 into the vendor directory, everything works as expected.

Tags rails, ruby,

New rails weblob

Posted by Chris Roos Sat, 05 Aug 2006 00:47:00

he he he

Tags blog, rails,

Helpful error messages when active record objects fail validation (in tests)

Posted by Chris Roos Wed, 02 Aug 2006 13:47:00

I posted a while back about displaying the reasons for an active record model failing validation.

I knew the code was hacky and nasty but didn’t realise quite how bad until I noticed how simply (1 line) the same thing was implemented in Rails 1.1 (and above).

Although it’s only useful for people still on Rails 1.0, I made a quick plugin that backports this change. I don’t really want to put it in my subversion repository, so have just pasted the necessary code below.

# File init.rb
require File.dirname(__FILE__) + '/lib/record_invalid'

# File record_invalid.rb
module ActiveRecord

  class RecordInvalid < ActiveRecordError
    def initialize(record)
      @record = record
      super("Validation failed: #{@record.errors.full_messages.join(", ")}")
    end
  end

end

To create a plugin from this, create the following structure in your rails_app/vendor/plugins directory.

active_record_validation_error/
|
|--init.rb (copy contents of code above)
|
|--lib/
   |
   |--record_invalid.rb (copy content of code above)

Tags activerecord, plugin, rails, validation,

Some realisations about testing

Posted by Chris Roos Mon, 17 Jul 2006 09:19:00

Having been predominantly involved in Rails development just recently, fixtures have taken on a slightly different meaning to what I now believe to be the original understanding. Having recently re-read Test Driven Development: By Example I re-discovered the ‘original’ meaning of the word (essentially shared objects used by all tests in a test case).

This has made me think about the way that I’ve been structuring my tests (or not structuring them as the case may be). I have never been a fan of the setup and teardown methods provided to us in xUnit and I now believe this to be due to my misunderstanding of the term fixture and therefore their intention. This has led to my test cases growing very large and containing many disparate tests. If there was any separation of the testcase then it was usually based on functionality of a particular method (which separately leads me to believe that they may have been better expressed in a new object). I now understand that test cases should be split to contain tests that rely on a common test fixture. As soon as I started to do this, the setup and (to a lesser degree) teardown methods have become much more useful.

I’ve also started to create my test cases as anonymous classes derived from Test::Unit::TestCase.

Class.new(Test::Unit::TestCase) do
  def test_
    assert true
  end
end

I find that this helps when trying to determine where a new test should be placed. If the class has a name I’m tempted to scan that and make a decision as to whether a new test belongs in this case. By keeping them anonymous, I have to inspect the test fixture in order to understand whether a new test belongs here.

Note I may also add a comment right below Class.new to explicitly specify the fixture for this testcase. This may seem to go against not naming the classes in the first place but I think part of the problem with class names is the lack of punctuation and therefore the difficulty in clearly expressing the test fixture.

I’ve chosen a somewhat contrived example to demonstrate my new found test structure.

Class.new(Test::Unit::TestCase) do
  # An empty stack

  def setup
    @stack = Stack.new
  end

  def test_should_accept_an_item_when_sent_push
    assert_nothing_raised { @stack.push(1) }
  end

  def test_should_complain_when_sent_top
    assert_raise(StackUnderflowError) { @stack.top }
  end

  def test_should_complain_when_sent_pop
    assert_raise(StackUnderflowError) { @stack.pop }
  end

end

The fixture for this test is an empty stack. The individual tests prod and query the empty stack to ensure that it behaves as expected. Although the example above is incredibly simple, I have consistently found that my individual test methods become much smaller.

Notice that the names of the tests all start ‘test_should’. Not only does this help us think about the responsibilities of the object; but it enables us to maintain living system specifications based on the tests. I believe this idea first came from testdox for Java but was introduced to me by Ben at work. In fact Ben has written a cool Agiledox server that displays the system specifications for your rails project.

The realisations set out above have come at around the same time as my taking more of an interest in rSpec (the stack test above has been adapted directly from one of the rSpec examples); a behaviour driven development framework for ruby. I haven’t done any real work with rSpec yet but am hoping it might improve the way I think of both testing and OO design in general.

Tags agiledox, bdd, rails, rspec, ruby, tdd,

Textmate command to display Active Record column attributes

Posted by Chris Roos Fri, 14 Jul 2006 08:18:00

I find myself constantly visiting either the console or the mysql command line to check out the attributes (columns) of active record objects.

I know there is a plugin by Dave Thomas that places the table structure within the model code itself. To be honest I’ve never tried this and it might well suffice (although I’m not 100% convinced I like the idea).

Anyhoo, I just spent some time whilst on the train hacking together a basic textmate command that displays a tooltip containing the table structure. It’s very basic (though you wouldn’t have thought it based on the amount of time it took me together¹); but seems to work ok.

The code is below. You’ll need to add a ruby command in the ruby textmate bundle, setting the intput to ‘selected text’ or ‘nothing’, the output to ‘show as tooltip’ and preferably a hotkey that isn’t one of the ones you use every day (however tempting that is).

#! /usr/local/bin/ruby

project = ENV['TM_PROJECT_DIRECTORY']
word = ENV['TM_CURRENT_WORD']

require "#{project}/config/boot"
require "#{project}/config/environment"

klass = Object.const_get(word) rescue nil
if klass and klass.class == Class and klass.ancestors.include?(ActiveRecord::Base)
  columns = klass.columns_hash

  data = []
  data += [%w[column primary sql_type default]]
  data += [%w[------ ------- -------- -------]]
  data += columns.collect { |col, attrs| [col, attrs.primary.to_s, attrs.sql_type.to_s, attrs.default.to_s] }

  STDOUT << data.inject('') do |output, array| 
    output + array.inject('') { |row_str, value| row_str + value.ljust(20) } + "\n"
  end
elsif klass and klass.class == Class and not klass.ancestors.include?(ActiveRecord::Base)
  STDOUT << "#{word} is not an Active Record derived class"
else
  STDOUT << "#{word} was not recognised as a class"
end

It relies on you having your entire rails project (the folder right above app, config, script etc) open in textmate (in order to find the config files). With that, just click on the name of an active record class (it has to be the actual class name at present, it’d be cool if it also worked based on the class definition you are currently in too) and hit your hotkey of choice (best to use the one you assigned to this command). Hey presto, a tooltip containing your table structure.

One odd thing I’ve noticed is that it never recognises standard class constants. It should display ‘this is not an active record derived class’ but instead always displays ‘was not recognised as a class’. Oh well.

Anyone find this remotely useful or want to finish off what I’ve started?

[1] I spent ages trying to develop within the textmate bundle editor before realising it’d be a helluva lot quicker and easier to develop in textmate proper. Hmmm.

Tags activerecord, rails, ruby, textmate,

Using in memory Active Record objects with associations for testing

Posted by Chris Roos Wed, 12 Jul 2006 01:23:00

Hitting the database in testing is bad.

Something that I’ve found relatively useful at work is to use real Active Record objects without any persistence (basically use AR#new and not AR#save[!] or AR#create[!]). When using standard techniques to create the association objects in memory you only get one side of the association at a time. Consider the following.

class Person < ActiveRecord::Base
  has_many :addresses
end
class Address < ActiveRecord::Base
  belongs_to :person
end

# One sided relationship with parent getting access to child
chris = Person.new
home = chris.addresses.build

chris.addresses.first
=> #<home>
home.person
=> nil

# One sided relationship with child getting access to parent
home = Address.new
chris = home.build_person

chris.addresses
=> []
home.person
=> #<chris>

# The example above that creates the address first could also have been written as.
# chris = Person.new
# home = Address.new(:person => chris)

You can see that in each case, only one side of the relationship has access to the other. If we were persisting the objects then the relationship would be two way via a database lookup. We can get a two way relationship in memory; although it has a certain duplication smell about it. Assuming the same models as above, we can amend the above examples like so.

# First example
chris = Person.new
home = chris.addresses.build(:person => chris)

chris.addresses.first
=> #<home>
home.person
=> #<chris>

# Second example
home = Address.new
chris = home.build_person(:addresses => [home])

home.person
=> #<chris>
chris.addresses.first
=> #<home>

I’ve occasionally used this in place of stubbing methods but am still not 100% convinced of the definite benefits one way or another. I do have some definite disadvantages of stubbing methods (nothing exciting, they’ve almost certainly been noted by many others in the past) which I’ll note down at a later date.

Tags activerecord, learning, rails, ruby,

More helpful messages for Active Record validation errors in Rails testing

Posted by Chris Roos Wed, 21 Jun 2006 08:10:00

In testing (and production code) we always try to persist Active Record models with the bang variants (save! and create!). This allows us to fail fast if any of our models fail our validation rules.

The problem is that the standard test/unit reporting framework displays the exception without any details of why the validation failed.

 1) Error:
test_should_fail_validation(AardvarkTest):
ActiveRecord::RecordInvalid:ActiveRecord::RecordInvalid
   /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/validations.rb:711:in 'save!'
   /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/validations.rb:673:in 'create!'
   /Users/chrisroos/dev/example_rails/test/unit/aardvark_test.rb:22:in 'test_should_fail_validation'

1 tests, 0 assertions, 0 failures, 1 errors

Note. This isn’t the fault of test/unit. It has to deal with generic exceptions and so cannot ‘know’ that Active Record Exceptions can be used to find the reason for the failure.

At this point, it would be very easy to jump into the code of the Active Record model, check out the validations and satisfy them within your code. However, it’d be much cooler if we could just add the reason for the failures to the error message that is displayed.

I initially played around with a method that took a block and would wrap the block with a begin/rescue clause to explicitly catch RecordInvalid exceptions, displaying the reason for failure.

Then I thought it’d be cool if we could integrate slightly more with both test/unit and rails. The result is a plugin (source) that can be installed as follows.

script/plugin install svn://dev.seagul.co.uk/plugins/test_unit_active_record_errors

The output from a failed validation now includes the validation errors. Yay.

 1) Error:
test_should_fail_validation(AardvarkTest):
ActiveRecord::RecordInvalid: ActiveRecord::RecordInvalid
   /opt/local/lib//gems/1.8/gems/activerecord-1.13.2/lib/active_record/validations.rb:711:in `save!'
   /opt/local/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/validations.rb:673:in `create!'
   /Users/chrisroos/dev/example_rails/test/unit/aardvark_test.rb:22:in `__test_should_fail_validation_without_custom_error_handling'
   /Users/chrisroos/dev/example_rails/config/../vendor/plugins/test_unit_active_record_errors/lib/test_unit_active_record_errors.rb:29:in `__send__'
   /Users/chrisroos/dev/example_rails/config/../vendor/plugins/test_unit_active_record_errors/lib/test_unit_active_record_errors.rb:29:in `test_should_fail_validation'
   /Users/chrisroos/dev/example_rails/config/../vendor/plugins/test_unit_active_record_errors/lib/test_unit_active_record_errors.rb:27:in `test_should_fail_validation'
ActiveRecord Errors:Surname can't be blank, Email can't be blank

1 tests, 0 assertions, 0 failures, 1 errors

The code is pretty hacky and includes no tests. It’s only been tested on Mac Os X 10.4.6 with Ruby 1.8.2 and Rails 1.0.0 but please feel free to do what you will with it if you think it might be of some use to you.

Tags active, rails, record, ruby, testing, validations,

Rails plugin to utilise Sqlite for testing

Posted by Chris Roos Fri, 02 Jun 2006 16:41:00

I feel pretty honoured that Geoffrey Grosenbach, of Rails podcast fame (plus so much more) has used the stuff I wrote a while back to create a rails plugin, enabling the easy use of sqlite for testing.

Tags rails, ruby, sqlite, testing,

Using active record models in rails migrations

Posted by Chris Roos Fri, 12 May 2006 16:32:00

I seem to remember that we first came across the “migrations out of sync with code” problems around the time that Scott Laird posted about it regarding typo.

Having just read Ben’s post about rails reliability, I was reminded again about the pitfalls of using model code in migrations.

“Migrations can be problematic if the code under which the migration was written changes before the migration is deployed and released.”

We’re a small team working on the same codebase, yet we still get bitten relatively frequently. A while back we started to re-define our AR classes within our migrations to ensure that we weren’t relying on model code. This has been documented elsewhere. The typo codebase has gone slightly further and created a BareMigration module that takes care of some magic for you. To be fair, I haven’t looked in detail so am not entirely sure what it does..

The problem with re-defining models in the migrations is that it relies on you remembering to re-define them (I’m good at ‘forgetting’ to do things unless I really have to). There is nothing to stop you using the real models; that is, until someone complains that the migrations are broken and you have to fix it.

We are currently using a slightly more robust (I think) solution that forces you to re-define any AR models that you wish to use in migrations. It does this by hijacking the ‘zero’ migration (a migration starting with 0 that gets loaded along with all other migrations but doesn’t interfere in any other way) to ensure any AR derived classes are defined in a module. It’s naive but seems to work ok.

If you don’t already have a zero migration, then create one. The name is important as you will need to create an empty class of the same name in the file itself (for migrations to be happy). Ours is called 000_rails_ext.rb. The code is below. Our code isn’t identical (yet) as I just realised that we had an ineffective call to super in the inherited method, now replaced with alias_method.

  class RailsExt
  end

  class ActiveRecord::Base
    class << self
      alias_method :__original__inherited, :inherited
      def inherited(base)
        raise "You're trying to use an ActiveRecord::Base derived class (--#{base}--) that is not defined in this migration." unless base.to_s =~ /^Migration.*/
        __original__inherited(base)
      end
    end
  end

  Dispatcher.reset_application!

The inherited callback is triggered everytime we define a class that inherits from ActiveRecord::Base. All we do is check that its name starts with Migration and raise an error if not (failing fast is good). The name is the fully qualified name and so includes namespaces. This is good as it allows us to place all re-defined classes in a module named Migration<number>. This module can then be included in the migration class itself. Although I can’t remember exactly now, I think the reason we need to reset_application! is because we have some calls to our models in environment.rb. These models would already be loaded and therefore any future references to them (in the migrations) would bypass our custom inherited method. By resetting the app, we are sure that we have no AR models loaded.

To re-define the models in the migration, we use something like..

  class MyFirstMigration < ActiveRecord::Migration
    def self.up
    end
    def self.down
    end
  end

  module Migration001
    class Person < ActiveRecord::Base
    end
  end
  MyFirstMigration.send(:include, Migration001)

Although sending the migration class the include message with the module isn’t great, we use it as it allows us to re-define all our classes below the actual migration. If we were to place the module at the top then this wouldn’t be required but I feel that it gets in the way of being able to read the migration clearly.

A problem with this method and re-defining classes directly in the migration

Both methods create AR derived classes in a namespace that isn’t the root namespace. We came across a problem where we were serializing the contents of one of these re-defined classes to the database from within our migration. When we came to read the serialized data in our code, it didn’t ‘know’ what a Migration001::Person was and so raised an exception. Without thinking about it too much, I think this might cause problems with STI too (I’ve seen notes to this effect from the Typo migration solution).

Tags migrations, rails, ruby,

In memory sqlite database for rails testing

Posted by Chris Roos Wed, 08 Feb 2006 08:54:00

Update 12th June 2006

There is now a plugin from Geoffrey Grosenbach that encapsulates this hack.

I finally took some time to get the sqlite in memory database working correctly for testing Rails apps.

It appears to be available by default (appearing in the example database.yml file) as

sqlite3_in_memory_example:
  adapter: sqlite3
  database: ":memory:"

Unfortunately, it doesn’t appear to be quite as easy as just replacing sqlite3_in_memory_example with test… The main problem being that in-memory databases have a tendency not to hang around (odd that).

My solution (which has every chance of being bad or downright incorrect) is to force the creation of the database in memory at the beginning of the tests.

The current implementation relies on either schema.rb being present or comprehensive migrations (i.e. you can use migrations to get a database from zero to your current development status).

If you wish to use the schema.rb approach then uncomment the following line in your environment.rb file.

config.active_record.schema_format = :ruby

Un-commenting this option ensures that the database schema is exported to schema.rb rather than adapter specific sql (ENV_database.sql).

Add the following code right below the ”# Include your application configuration below” line in environment.rb.

def in_memory_database?
  ENV["RAILS_ENV"] == "test" and 
  ActiveRecord::Base.connection.class == ActiveRecord::ConnectionAdapters::SQLiteAdapter and
  Rails::Configuration.new.database_configuration['test']['database'] == ':memory:'
end

if in_memory_database?
  puts "creating sqlite in memory database"
  load "#{RAILS_ROOT}/db/schema.rb" # use db agnostic schema by default
#  ActiveRecord::Migrator.up('db/migrate') # use migrations
end

The reason for adding this code at this point and not just at the end of the environment file is that you may have further logic in environment that actually relies on the database being present.

If you are running your tests using rake then it shouldn’t matter whether you use the schema or migrations to create the database. This is because the rake test tasks rely on the test database being set-up, which involves exporting the development db schema to file.

If you wish to run tests individually (which, of course, you do) and you choose to use the schema to create your test scructure, you must make sure that it first exists. The easiest way is to run the following rake task.

rake db_schema_dump

That should be all there is to it; although I must confess that I’ve seen additional strange problems when running certain tests against the sqlite in-memory database. In certain cases I think these may be fixture related. Where, in MySql, you seem to be able to omit required fields (not null) from a fixtures file, sqlite requires their presence. In other cases; who knows…

Testing your sqlite installation

One final note that may be of interest. When I first started playing with sqlite3 with Ruby it seems that I didn’t have it installed correctly (relying on pre-installed sqlite and the sqlite3-ruby gem). As per this article I had to un-install sqlite3-ruby gem and first install the swig port before re-installing sqlite3-ruby gem. You can test for the correct installation by running the Active Record tests against sqlite by following these steps.

Change directory to Active Record tests (rails/activerecord/test)

Run base_test.rb, supplying the location of the sqlite3 connection to the load path.

ruby -I "connections/native_sqlite3/" base_test.rb

You can also run the tests using the sqlite3 in memory connection. To do so, perform these actions before running the tests as above.

Remove the standard sqlite3 connection

rm connections/native_sqlite3/connection.rb

Rename in_memory_connection.rb as connection.rb

mv connections/in_memory_connection.rb connections/connection.rb

I’d suggest doing this against an svn checked out copy of rails, purely due to the these at which the changes can be reverted.

Useless stats

Changing to in-memory testing shaved about 33% off the running time of our unit tests.

Tags rails, ruby, sqlite, test,