Simon Menke

Milkshakes for everyone

2009-09-30T00:00:00-07:00

I’m tired of Custom CMS systems!

In the past few years I’ve build dozens of custom CMS systems. Nearly all of them were written in Rails. And now they are more of a problem than a solution. They all do things in a slightly different way. and there is a lot of duplication across these applications. Not because of the lack of plugins but because all these plugins require you to write or even generate code yourself. The reason why most plugins are written this way is obvious but I still need to do too much.

I hate prefab CMS systems!

I’ve looked at many prefab CMS systems, trying to find a simple solution that would do all the things I needed to do and more importantly all the things our clients needed to do. Another problem with prefab CMS systems is customizability. Some systems like Radiant are fairly customizable but simply too hard for our clients. Others, like Browse CMS, look nice but fail when compared to our wish list.

What I needed.

I wanted a system that was extremely modular, highly customizable and very client friendly. Unfortunately I didn’t find it so I build it myself. Currently most of it is still under wraps but most of the modules/gems/plugins (whatever you would like to call them) will be released under one or an other OSS license. Starting with the core today.

What I made.

The core is called Milkshake and it builds composite Rails applications.
Composite Rails applications are Rails applications that are build using many smaller Rails applications which in there turn can be build using many even smaller Rails applications.

A composite Rails application.
Each node represents a fully functional rails application (except for Paperclip, Globalize, MetaController and MetaERB)

Essentially Milkshake automatically loads gem-plugins, manages migrations and links static assets. This allows any regular gem plugin to become as privileged as any standard Rails application. Milkshake also allows you to take any regular Rails application and build a gem from it.

A list of what Milkshake does.

Automatically configure/load dependencies defined in config/milkshake.yml.
Automatically migrate the database on relink.
Make database snapshots before migrating.
Load locale files from gems in rails > 2.3.4.
Initializers for gems like config/initializers but in rails/initializers.
Symlink public directory from gems into public/vendor/[GEM_NAME].
Extract data and configuration directories and files from the rails directory.

Installing Milkshake.

Installing Milkshake is easy but make sure you installed gemcutter.

# only if you don't already have gemcutter installed.
$ sudo gem install gemcutter
$ sudo gem tumble
# now install milkshake
$ sudo gem install milkshake

Building a gem with Milkshake.

# generate an empty gem app
$ milkshake create.gem my_gem
# answer the questions
$ cd my_gem
$ ls
README    app     db    lib   public  script  tmp
Rakefile  config  doc   log   rails   test    vendor

Now you have a regular rails application with some extra files and directories needed for packaging everything up in a gem.

rails/init.rb: This is the regular Rails gem plugin init file.
rails/initializers: In here you can put initializers like in the config/initializers directory. Note that the config/initializers directory doesn’t get packaged.
config/milkshake.yml: In this file you define your gem dependencies. This replaces the config.gem directive in the config/environment.rb file.
config/preinitializer.rb: This file loads the milkshake code and extends Rails.
lib/tasks/jeweler.rake: This rake file contains the Jeweler tasks for packaging the Rails app.

If you look a little deeper you’ll notice that the application_controller.rb and the application_helper.rb are missing. Generally you only add these to the top level gem but you don’t have to as Milkshake has a fallback application_controller.rb which get loaded if you don’t include your own.

After writing some code, adding some migrations and some static assets you can build this rails app like so:

$ rake version:write # only the first time
$ rake gemspec build

A new directory called pkg gets created, it contains the build gems. To install you new gem just do a gem install [pathtogem] or:

$ rake install

Using a Milkshake gem.

# generate an empty host app
$ milkshake create.host my_host
$ cd my_host
$ ls
config  db  log public  script  tmp

As you can see a host app is mush slimmer than a gem app. The reason for this is that you don’t use host apps for development and they should not contain code. They are just used for serving your composite app. Edit the config/milkshake.yml file.

gems:
  my_gem: {} # no special lib, no specific version requirement, no specific source

Next touch the tmp/relink.txt file to make sure Milkshake links all the gems into the host app. Note that you need to restart the host app in order to actually run linker. Also note that (re-)linking the host app automatically runs all the migrations.

$ touch tmp/relink.txt
$ touch tmp/restart.txt # restart the web server (in this case passenger)

The way Milkshake alters the Rails initializer.

Milkshake adds some phases to the Rails initialization process. Below you can see all the added phases compared to a regular Rails initialization process.

Rails	Rails with Milkshake
require frameworks	require frameworks
–	extend frameworks with milkshake
load environment	load environment
initialize database	initialize database
–	take database snapshot
–	migrate database
load gems	load gems
load plugins	load plugins
–	load gem initializers
load application initializers	load application initializers

Database agnostic Snapshots

2009-09-28T00:00:00-07:00

Why and how I use production data in our development environment.

At Mr. Henry we like to use real production data in our development environment. It allows us to see how our clients use our applications and how we can improve on that. We also like Sqlite3 in development and MySQL in production.

The problem I faced when making an SQL dump is the way Rails treats each type of database. For instance in Mysql booleans are represented as 0 and 1 but in Sqlite3 f and t are used. This is where Snapshots comes in. Snapshots allows us to dump a database’s schema and its data in a Ruby file. This file looks similar to the db/schema.rb file, in fact the first section is identical. The second section is the most interesting though, it contains the actual data.

Here’s an example of what such a file might look like.

# This file was generated by the Snapshots::DatabaseDumper and
# can be loaded with the Snapshots::DatabaseLoader.

ActiveRecord::Schema.define do

  create_table "schema_migrations", :id => false, :force => true do |t|
    t.string "version", :null => false
  end

  add_index "schema_migrations", ["version"], :name => "unique_schema_migrations", :unique => true

  create_table "posts", :force => true do |t|
    t.string   "title"
    t.text     "content"
    t.boolean  "published"
    t.datetime "created_at"
    t.datetime "updated_at"
  end

end

Snapshots::DatabaseLoader.load do

  for_table "posts" do |t|
    t.insert(
      :id => 1,
      :title => "Hello World",
      :content => "My body text...",
      :published => false,
      :created_at => "Wed, 28 Sep 2009 09:24:04 +0000".to_time,
      :updated_at => "Wed, 28 Sep 2009 09:24:04 +0000".to_time
    )
    t.insert(
      :id => 2,
      :title => "Database Agnostic Snapshots",
      :content => "You are reading it...",
      :published => true,
      :created_at => "Wed, 28 Sep 2009 09:24:26 +0000".to_time,
      :updated_at => "Wed, 28 Sep 2009 09:24:26 +0000".to_time
    )
  end

  for_table "schema_migrations" do |t|
    t.insert(
      :version => "20090928091802"
    )
  end

end

As you can see this file contains just Ruby. One strange thing you might notice is the schema_migrations table. It is included in order to preserve the migration state.

How to install and use Snapshots.

Installing Snapshots is easy but make sure you installed gemcutter.

# only if you don't already have gemcutter installed.
$ sudo gem install gemcutter
$ sudo gem tumble
# now install snapshots
$ sudo gem install snapshots

Next go to one of you rails apps and make start making snapshots.

# dump a new snapshot
$ snapshots dump
# list all you snapshots.
$ snapshots list
1253528651 @ Wed Sep 28 09:28:11 +0200 2009 : db/snapshots/snapshot_1253528651.rb
# load a particular snapshot using the snapshot id.
$ snapshots load 1253528651
# or using a filename (not a snapshot doesn't need to be in the db/snapshots directory).
$ snapshots load db/snapshots/snapshot_1253528651.rb

How to use Snapshots from within Ruby/Rails.

Snapshots also has simple ruby API which can easily be used from within your rails application.

require 'snapshots'

# find all snapshots.
Snapshots.find.each do |p|
  puts p
end

# dump a new snapshot
path = Snapshots.dump

# load a snapshot
Snapshots.load(version_or_path)

Note on Patches/Pull Requests.

Fork the project.
Make your feature addition or bug fix.
Add tests for it. This is important so I don’t break it in a future version unintentionally.
Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
Send me a pull request. Bonus points for topic branches.
Yes, I know this is the Jeweler scaffold :)

Navigate between tabs like in TextMate

2009-04-26T00:00:00-07:00

I like the way TextMate lets me navigate all my open tabs using ⌘← and ⌘→. So here is a small tutorial on setting this up on your computer.

Click the add button

On tab to the right

Don’t forget the other shortcut.

Introducing GM [OUTDATED]

2009-01-18T00:00:00-08:00

Introduction / Motivation

The problem with the existing tools for building gems, like hoe and newgem, is they generate a whole host of files which are hard to maintain. Tools like gemify and gemhub on the other hand are to simplified and work only for very small gems which follow the imposed conventions.

The solution would be to make a tool which doesn’t generate unneeded files and uses a simple configuration format like YAML. Enter GM.

GM is a framework for building gems in a clean and simple way. It uses YAML as its configuration format and generates only the absolute minimum needed to get you going. GM is also designed to be extendable in (almost) every way.

How to use GM

Install GM

sudo gem install simonmenke-gm

Global configuration

Put the following snippet in ~/.gmrc

author:
  name:  [Your Name Here]
  email: [Your Email Here]
github:
  username: [Your GitHub Username Here]

Create a gem

gm create name_of_your_gem

Gem configuration

Put the following snippet in the Gmfile of your gem.

general:
  name:     [The Gem Name]
  version:  [The Gem Version]
  summary:  [A Summary]
  homepage: [The Homepage Of The Gem]
dependencies:
  - [Name Of Gem]  [Version Requirement (>= 1.0.0)]

Write code

I can’t help you here ;)

Build or install your gem

gm build
gm install

Dump a .gemspec file for GitHub

gm spec

Known problems with GM

The gm command may conflict with the gm tool from GraphicsMagick. To bypass this use gem-make instead of gm.

Please post any bug reports or feature requests here