Frontended.com

Legacy Migrations is a little gem that helps you migrate data from an old database structure to a new db and db structure. It keeps a status report for failed validations, and also lets you update data if all you need is a nightly data update. Here’s a quick sample that transfers data from the model Person to the model Animal (assuming the models are subclasses of either ActiveRecord::Base or CSV::Table):

require 'legacy_migrations'

#Assuming you've already created classes for source and destination tables
transfer_from Person, :to => Animal do
  match_same_name_attributes :only => [:sex, :age]
  from :name, :to => :pet_name
  from :from_record, :to => :species do |from_record|
    "Homo Sapien"
  end
end

Here’s the full list of options you can pass to ‘transfer_from’ (at the time of writing, of course):

• :limit – Set a limit to the number of rows to transfer. This is useful when you’re trying to find faulty data in the source table, and don’t want to run the entire dataset.
• :validate – Default = true. Use ActiveRecord validations when saving destination data. Reports errors in the StatusReport (a singleton object)
• :source_type – Default = :active_record. Sets the source destination type. Options: :active_record: Assumes the From option is a class that inherits from ActiveRecord::Base, then iterates through each record of From table by using From.all.each… :other: Assumes the From option is an iterable ‘collection’ whose elements/items can respond to all columns speficied in the given block.
• :store_as – Store the resulting generated record to be used in a future transfer_from or update_from operation. You can access this object like so:

  transfer_from Person, :to => Species, :store_as => 'new_species' do
    from :name, :to => :species_name
      "Homo Sapien"
    end
  end
  
  transfer_from Person, :to => Animal do
    from :pet_name, :to => > :name
    stored :new_species, :to => :species  # Uses the species that we
                                          # created in the previous
                                          #  transfer_from
  end
  

Aside from those helpers, you also get the ‘match_same_name_attributes’ helper that…wait for it…matches same-name columns between the source and destination tables. Here’s a quicky:

transfer_from Person, :to => Species, :store_as => 'new_species' do
  match_same_name_attributes :only => [:name, :age, :date_of_birth]
  # There's also an 'except' option, and if 
  # you don't use either, then the script
  # just transfers all same-name columns.
end

You can also use the gem to update data periodically using the ‘update_from’ operation. The
update_from operation uses the well-known squirrel plugin syntax to match records from the source database, to records in the destination database. If the source record doesn’t exist in the destination, then it gets automatically created. Think of this as ‘find_or_create_by’ on steroids. For example, let’s say you want to update first names in the Animal model, from the latest data in the People model, and the identifying and matching features between the source and destination tables are the date_of_birth and last_name columns:

update_from Person, :to => Animal do
  based_on do |from_record|
    date_of_birth == from_record.dob
    last_name ==  from_record.last_name
  end

  from :first_name, :to => :first_name
end

Here’s the git repo:

http://github.com/btelles/legacy_migrations

To install it just use:

gem install legacy_migrations

If you have any suggestions, please let me know!

Update:

By the way, this gem is considered beta software and to get a better idea for how to use it, you may want to look at the rspec files. They’re the most comprehensive documentation.

Here are the slides to a quick presentation I gave at our local Ruby Brigade, in case anyone’s interested.