It’s Time To Double Up (Using Amazon’s RDS Read Replication Database Servers With Heroku For Master-Slave Replication)

Heroku is great for rapid application development but if you want to run multiple databases it doesn’t provide any options. Running multiple databases in a master-slave orientation can provide an elegant solution to many scaling issues. This can be accomplished on heroku using my forked version of schoefmax’s gem multi_db.

First a quick look at master-slave replication

This database setup utilizes one or more read database(s) that only service read requests, and a master database that takes care of the writes, this provides a more manageable approach to your application’s database needs. This is advantageous for applications that require rapid response from read requests, as they are not hindered by any write requests that may be taking place. This diagram illustrates the process.

As you can see mysql takes care of the replication, a process made even easier by Amazon’s RDS Read Replicas.

-Integration-

Build a Read Replica

Setup for this is pretty painless. You’ll just need to create a read replica of your RDS instance. Note that this will take a snapshot of your master database and cause a little downtime.

This read replica should be the same size or larger than your master database.

Install Gem

Next you’ll need to clone the multi_db gem into your vendor/gems directory.

git clone git@github.com:plaxis/multi_db.git

Don’t forget to add the version.

mv multi_db/ multi_db-0.2.2

Source the gem in your Gemfile like so.

gem "multi_db", "0.2.2", :path => "vendor/gems/multi_db-0.2.2"

This forked version of the multi_db gem has a modified init_slaves method in the ConnectionProxy class. This method looks in the gem itself for the slave database credentials.

def init_slaves
        returning([]) do |slaves|
          YAML.load_file("vendor/gems/schoefmax-»
multi_db-0.2.2/lib/db.yml").each do |name, values|
            if name.to_s =~ /#{self.environment}_(slave_database.*)/
              weight  = if values['weight'].blank?
                          1
                        else
                          (v=values['weight'].to_i.abs).zero?? 1 : v
                        end
              MultiDb.module_eval %Q{
                class #{$1.camelize} < ActiveRecord::Base
                  self.abstract_class = true
                  establish_connection(YAML.load_file»
('vendor/gems/schoefmax-multi_db-0.2.2/lib/db.yml')['#{name}'])
                  WEIGHT = #{weight} unless const_defined?('WEIGHT')
                end
              }, __FILE__, __LINE__
              slaves << "MultiDb::#{$1.camelize}".constantize
            end
          end
        end
      end

Now you’ll need to configure your slave databases in vendor/gems/multi_db-0.2.2/lib/db.yml. Be sure to stick to the naming conventions, the database name must match the environment. Note that only the slaves need to be declared here, the database declared by Heroku will be used as the master database, this can be seen with “heroku config”.

production_slave_database:
  adapter: mysql
  database: myapp_production
  username: root
  password:
  host: 10.0.0.2
  weight: 1

If you wish to control the distribution of queries to each slave you can assign a weight. A database with weight: 1 will receive half the queries as a database with weight: 2. A contribution by the brilliant David Palm.

Add this to the environment file that you wish to run the slave databases in.

config.after_initialize do
  MultiDb::ConnectionProxy.setup!
end

You should be good to go! Feel free to leave a comment with any questions or suggestions and I’ll try and reply as quickly as possible and as long as i know the answer.ROFL. Thanks!