The Relicans

Kirk Haines
Kirk Haines

Posted on • Updated on

Departure 6.3.0 Released!

Departure

Departure is an adapter for Rails that automatically routes ActiveRecord migrations through the Percona Migrator, aka pt-online-schema-change. This allows migrations on MySQL family databases that might otherwise lock the table to be done without locks and without downtime.

The 6.3.0 version was released today, offering support for your Rails 6.1 applications.

The pt-osc tool runs migrations by applying them to a copy of the original table, using a trigger on the original table to ensure that anything that happens to it also happens to the copy during the time that the migration is running. Once the original table is fully migrated, the tool does an atomic swap of the now-migrated copy of the original table with the original, and then it cleans everything up for you.

This allows even very large tables to have migrations ran on them that might otherwise lock the database for minutes or hours to be handled without service impacting downtime for your userbase, and without requiring that your developers depend on an SRE or other external resource to run the migration for them.

These migrations are run in a safe manner, on a copy of the original table, which means that running them is also very safe. If there is any problem that prevents the migration from completing, your original table is unaffected.

Usage

If your Gemfile includes the Departure gem, it will be used for all migrations (though there are mechanisms to fine-tune this, if you need to).

gem "departure", "~> 6.3"
Enter fullscreen mode Exit fullscreen mode

Developers can write migrations that are run via departure just like they would write any other migration. There are no special considerations of peculiarities of syntax that they must follow.

class ChangeLowercaseEmailLimitAndCollation < ActiveRecord::Migration[6.1]
  def up
    change_column :credentials, :lowercase_email, :string, limit: 255, collation: "utf8mb4_bin"
  end

  def down
    raise ActiveRecord::IrreversibleMigration
  end
end
Enter fullscreen mode Exit fullscreen mode

Likewise, there is no change to how migrations are run:

rake db:migrate
Enter fullscreen mode Exit fullscreen mode

To use Departure, you will have to have the Percona Toolkit installed. There is ample information online for installing it for your operating system, but most questions can probably be answered by looking at the official Percona documentation:

https://www.percona.com/doc/percona-toolkit/LATEST/installation.html

Customization

The pt-online-schema-change tool has a large number of options that can be used to fine-tune its behavior, allowing one to influence both how it runs, and how aggressively it uses the available resources to run the migration. In some cases, one may want to change some of these options from the defaults, such as choosing to be less aggressive with resources in order to avoid service impacting slowdowns while running migrations, or to run an external utility that alters operations (this little plugin is very useful) ).

For example:

PERCONA_ARGS='--chunk-time=1 --critical-load Threads_running=55' bundle exec rake db:migrate:up VERSION=xxx
Enter fullscreen mode Exit fullscreen mode

In this example, it assigns several additional parameters to the PERCONA_ARGS environment variable. The contents of that variable are applied to the invocation of pt-online-schema-change as additional command-line options.

This example specifies a couple of options:

1) --chunk-time=1 -- This specifies that pt-osc is to use no more than one second on any one data copy operation. It will dynamically adjust the number of rows that it copies in each chunk to ensure that it stays under the time limit specified.
2) --critical-load Threads_running=55 -- pt-osc will examine the output of the MySQL command SHOW GLOBAL STATUS after every chunk, and it will abort the migration if the specified value is exceeded. So, in this case, if the database has more than 55 running threads, the tool will abort the migration. This can be used to fine-tune a migration in order to ensure that it makes the most of the available resources without overwhelming the database. By default, pt-osc aborts if there are more than 50 running threads.

The other mechanism for tuning the operation of the tool is to do it in code. Options can be specified via global_percona_args during the tool's configuration.

Departure.configure do |config|
  config.global_percona_args = '--chunk-time=1 --critical-load Threads_running=55'
end
Enter fullscreen mode Exit fullscreen mode

A useful way to leverage this capability is to implement it within a separate gem that is included in your project's dependencies. This lets you ensure that the same set of configurations are used across all of your projects, and it lets you adjust them by making a release to a small gem instead of requiring an update within the project itself.

Below is a variation of a real-world gem that I once wrote for this purpose:

require "departure"

module DepartureDefaults

  def self.pt_plugin_path
    File.join(__dir__,
              "departure_defaults",
              "pt-online-schema-change-fast-rebuild-constraints.pl")
  end

  def self.global_percona_args
    %W( --charset=utf8mb4
        --recurse=0
        --recursion-method=none
        --max-load\ Threads_running=80
        --critical-load\ Threads_running=120
        --chunk-time=1.0
        --plugin\ #{pt_plugin_path}
        --sleep\ 0.1
        --alter-foreign-keys-method\ rebuild_constraints).join(" ")
  end

  Departure.configure {}
  Departure.configuration.global_percona_args = global_percona_args

end
Enter fullscreen mode Exit fullscreen mode

The full list of configuration options can be found here:

https://www.percona.com/doc/percona-toolkit/LATEST/pt-online-schema-change.html

Selective Use of Departure

As is often the case, it may be that most of your migrations are already table-lock-safe, and thus there is no reason to run them through pt-online-schema-change. If you prefer to only run specific migrations through Departure and to let the rest of them be handled via the normal Rails migration handling code, Departure has your back.

First, set this in your configuration (such as in your configuration gem similar to the one above):

Departure.configure do |config|
  config.enabled_by_default = false
end
Enter fullscreen mode Exit fullscreen mode

With that configuration flag set to false, only migrations that are specially marked to use Departure will do so. This is done via uses_departure!.

For example:

class ChangeAReallyLongSlowUglyThing < ActiveRecord::Migration[6.1]
  uses_departure!

  def up
    # Here's your long, slow, ugly migration.
  end

  # Other stuff...
end
Enter fullscreen mode Exit fullscreen mode

Contributing

We welcome contributions to Departure. If you use it, and you find bugs, or you use it, and you think something could be improved, I welcome you to help us to keep Departure useful to the community.

Fork the project, create a branch for your change, make your change, and don't forget to write appropriate tests for your change and to make sure that the suite of specs passes, and then make a pull request. We welcome it!


I stream on Twitch for The Relicans. Stop by and follow me at https://www.twitch.tv/wyhaines, and feel free to drop in any time. In addition to whatever I happen to be working on that day, I'm always happy to field questions or to talk about anything that I may have written.

Discussion (0)