Why and how I use production data in our development environment.
At Mr. Henry we like to use real production data in our development environment. It allows us to see how our clients use our applications and how we can improve on that. We also like Sqlite3 in development and MySQL in production.
The problem I faced when making an SQL dump is the way Rails treats each type of database. For instance in Mysql booleans are represented as 0 and 1 but in Sqlite3 f and t are used. This is where Snapshots comes in. Snapshots allows us to dump a database’s schema and its data in a Ruby file. This file looks similar to the db/schema.rb file, in fact the first section is identical. The second section is the most interesting though, it contains the actual data.
Here’s an example of what such a file might look like.
# This file was generated by the Snapshots::DatabaseDumper and
# can be loaded with the Snapshots::DatabaseLoader.
ActiveRecord::Schema.define do
create_table "schema_migrations", :id => false, :force => true do |t|
t.string "version", :null => false
end
add_index "schema_migrations", ["version"], :name => "unique_schema_migrations", :unique => true
create_table "posts", :force => true do |t|
t.string "title"
t.text "content"
t.boolean "published"
t.datetime "created_at"
t.datetime "updated_at"
end
end
Snapshots::DatabaseLoader.load do
for_table "posts" do |t|
t.insert(
:id => 1,
:title => "Hello World",
:content => "My body text...",
:published => false,
:created_at => "Wed, 28 Sep 2009 09:24:04 +0000".to_time,
:updated_at => "Wed, 28 Sep 2009 09:24:04 +0000".to_time
)
t.insert(
:id => 2,
:title => "Database Agnostic Snapshots",
:content => "You are reading it...",
:published => true,
:created_at => "Wed, 28 Sep 2009 09:24:26 +0000".to_time,
:updated_at => "Wed, 28 Sep 2009 09:24:26 +0000".to_time
)
end
for_table "schema_migrations" do |t|
t.insert(
:version => "20090928091802"
)
end
endAs you can see this file contains just Ruby. One strange thing you might notice is the schema_migrations table. It is included in order to preserve the migration state.
How to install and use Snapshots.
Installing Snapshots is easy but make sure you installed gemcutter.
# only if you don't already have gemcutter installed.
$ sudo gem install gemcutter
$ sudo gem tumble
# now install snapshots
$ sudo gem install snapshotsNext go to one of you rails apps and make start making snapshots.
# dump a new snapshot
$ snapshots dump
# list all you snapshots.
$ snapshots list
1253528651 @ Wed Sep 28 09:28:11 +0200 2009 : db/snapshots/snapshot_1253528651.rb
# load a particular snapshot using the snapshot id.
$ snapshots load 1253528651
# or using a filename (not a snapshot doesn't need to be in the db/snapshots directory).
$ snapshots load db/snapshots/snapshot_1253528651.rbHow to use Snapshots from within Ruby/Rails.
Snapshots also has simple ruby API which can easily be used from within your rails application.
require 'snapshots'
# find all snapshots.
Snapshots.find.each do |p|
puts p
end
# dump a new snapshot
path = Snapshots.dump
# load a snapshot
Snapshots.load(version_or_path)Note on Patches/Pull Requests.
- Fork the project.
- Make your feature addition or bug fix.
- Add tests for it. This is important so I don’t break it in a future version unintentionally.
- Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
- Send me a pull request. Bonus points for topic branches.
- Yes, I know this is the Jeweler scaffold :)