The Relicans

The Relicans is a community of amazing relicans

Recursively creating communities of developers; learn, create, teach, repeat.

Create new account Log in
loading...

Implementing a Ruby-like #send() in Crystal

Kirk Haines
I've had a 30+ year career across spectrum from System and DevOps to SRE type work to (a lot of) Software Engineering, and I love helping others to grow in their careers.
・9 min read

Just sit right back and you'll hear a tale
a tale of a fateful trip,
that started from this workstation,
aboard this tiny editor.

Recap: What Is send()?

Consider the following simple Ruby/Crystal class (it will work identically in both languages):

class AddIt
  def add(x,y)
    x + y
  end
end

adder = AddIt.new
adder.add(1,1) # -> 2
Enter fullscreen mode Exit fullscreen mode

This is the typical way to call a method. It is direct and declarative.

Ruby offers another option for method invocation. If one thinks of an object as something that can receive a message, and a method plus its parameters as a message, then it makes sense that you would be able to send messages to objects.

This is where the #send method comes into the picture. This method, which may also be called as #__send__, takes as its first argument the name of the method which is to be called. Subsequent arguments are passed to the named method as the arguments to that method.

So, given the previous class as an example, this would result in the same method invocation as in the first example:

add.send("add", 1, 1)
Enter fullscreen mode Exit fullscreen mode

There is no advantage to calling that method in that way, with a string literal for the method name, then there is in doing it the standard way. The real power of the #send method comes when the method to invoke is more dynamic, being provided in a variable.

This is frequently used in Ruby as part of its metaprogramming toolkit. In Rails alone, it is used dozens of times in the main codebase. Crystal, however, does not have a #send method.

Metaprogramming in Crystal

The TL;DR regarding Crystal is that while it shares a few of Ruby's mechanisms for metaprogramming, Crystal, being compiled, lacks both eval() and #send methods. Crystal's alternative for most metaprogramming tasks is to offer a macro language, which is essentially a subset of Crystal that can be used at compile-time in order to generate new code.

That is, macro code is used to generate code at compile-time, which is inserted into the code, in place of the macro code. This form of programmatic code generation is very powerful.

Macro code has access to deep introspection about the structure of the code. Among the wealth of information that macros can access is information on the methods that exist in a class, as well as the arguments that those methods are expecting, and the declared type information on those methods.

For example, imagine that you have a framework where users can subclass a master controller, to provide hooks for events. Inside that framework, you have a method that is supposed to route different events to the correct methods in the subclass. So, you want to write a case statement that can do this. You can use a macro to build that case statement:

macro dispatch(type, request, params)
  case {{ type.id }}
  {% for method in @type.methods %}
    {% if method.name =~ /^handle_/ %}
  when "{{ method.name.gsub(/^handle_/, "") }}" then {{ method.name.id }}({{ request.id }}, {{ params.id }})
    {% end %}
  {% end %}
  else
    raise NotImplementedError.new("handle_#{ {{ type.id }} }")
  end
end
Enter fullscreen mode Exit fullscreen mode

That macro will iterate the methods in the class in which it is called -- represented by @type. If the method name starts with handle_, it then writes a when line for the case statement that matches whatever follows handle_, and then calls the handler method.

As you can see, macro code can be very useful to write code, dependent on other internal code details, on our behalf.

So How Does This Relate to Implementing #send?

Imagine that you have a class like this:

class Foo
  def a(val : Int32)
    val + 7
  end

  def b(x : Int32, y : Int32)
    x * y
  end

  def c(val : Int32, val2 : Int32)
    val * val2
  end
end
Enter fullscreen mode Exit fullscreen mode

What if you wanted to manually create a #send method for that class? The key to implementing this would be to have some sort of a lookup table that you could store the method into, along with some way of storing those method calls.

Proc offers one way to do this.

Foo = ->(object : Klass, arg : String) {
  object.foo(arg)
}

Foo.call("something")
Enter fullscreen mode Exit fullscreen mode

Wrapping a method call in a proc lets one indirectly call that method, and procs have the advantage that they can be stored in a data structure like a NamedTuple or a Hash.

So, one could do something like this:

module SecretSauce
  SendLookupTable = {
    "Int32": {
      "a": ->(obj : Foo, val : Int32) { obj.a(val) },
    },
    "Int32Int32": {
      "b": ->(obj : Foo, x : Int32, y : Int32) { obj.b(x, y) },
      "c": ->(obj : Foo, val : Int32, val2 : Int32) { obj.c(val, val2) }
    }
  }

  def send(method, arg1 : Int32)
    SendLookupTable["Int32"][method].call(self, arg1)
  end

  def send(method, arg1 : Int32, arg2 : Int32)
    SendLookupTable["Int32Int32"][method].call(self, arg1, arg2)
  end
end
Enter fullscreen mode Exit fullscreen mode

Including that module in our Foo class will create overloaded #send methods that can be used to call the methods in our class.

foo = Foo.new
puts foo.send("a", 7)
puts foo.send("b", 9, 7)
Enter fullscreen mode Exit fullscreen mode

Making It a Little Faster

The basic pattern there works. It is a little slower than direct method calls, though.

foo = Foo.new

Benchmark.ips do |ips|
  ips.report("direct") {foo.b(rand(1000), rand(1000))}
  ips.report("send") {foo.send("b", rand(1000), rand(1000))}
end
Enter fullscreen mode Exit fullscreen mode

Running that with --release will return a result something like this:

direct 385.98M (  2.59ns) (± 1.64%)  0.0B/op        fastest
  send 230.79M (  4.33ns) (± 2.69%)  0.0B/op   1.67x slower
Enter fullscreen mode Exit fullscreen mode

That's not terrible, but is there a way to make it faster?

It turns out that there is. Crystal provides a macro, record, that provides a convenient syntax for defining a struct with a given name, properties, and, optionally, methods. We can use it kind of like Proc was used, to wrap up a set of arguments and a method call:

record Foo, object : Klass, arg : String do
  def call
    object.foo(arg)
  end
end
Enter fullscreen mode Exit fullscreen mode

That will define a struct that takes two arguments to initialize it. The struct will have one method defined on it, #call, which calls another method on the object that is passed to it upon creation. So, functionally, it does basically the same thing as the Proc. The difference, though, is in how the compiler optimizes those calls:

     direct 384.73M (  2.60ns) (± 2.24%)  0.0B/op   1.00x slower
  send proc 229.78M (  4.35ns) (± 2.83%)  0.0B/op   1.67x slower
send record 384.78M (  2.60ns) (± 1.93%)  0.0B/op        fastest
Execute:                           00:00:21.039084400 ( 105.39MB)
Enter fullscreen mode Exit fullscreen mode

More or less, there is no difference in performance between a direct method call and a call that runs through our hand-built #send that is using *record*s to make the method calls. That's pretty good!

OK, But How Can This Be a Generic Solution?

This is where everything loops back to macros. Looking back at the first example, where a case statement was being built. The core of that macro was a simple loop:

{% for method in @type.methods %}
#
#
#
{ %end %}
Enter fullscreen mode Exit fullscreen mode

Crystal provides all of the tools to write macros that can generate the same code that we built by hand. Referring back to the proof-of-concept, the lookup table was keyed by a text representation of the method type signature, and the send methods then use that same signature.

Data Share Lookup Table

That is enough of a starting point. The first thing that we need is to build a convenience table to allow easily repeated lookups of data that will be used in multiple places. Crystal macros allow access to constants, so the code can build a table inside of a constant for this purpose.

This macro will build a table that associates the arguments for a method with a simple, parsable text representation of that type signature that only uses characters that can be used in a constant name. It separates types in a union by a single underscore, and it separates the types for discrete arguments by a double underscore.

macro build_type_lookup_table
  # This table associates the method args with a simple, parseable text representation
  # of that type signature, separating individual argument's types with a double-underscore,
  # and types within a union by a single underscore.
  TypeLookupByLabel = {
  {% for method in @type.methods %}
    {{ method.args.symbolize }} => {{
                                     method.args.reject do |arg|
                                       # Reject everything that lacks a type restriction.
                                       arg.restriction.is_a?(Nop)
                                     end.map do |arg|
                                       arg.restriction.id.gsub(/ \| /, "_").id
                                     end.join("__")
                                   }},
  {% end %}
  }
end
Enter fullscreen mode Exit fullscreen mode

It also checks to ensure that the restriction exists, and skips the method if it does not. This is because Proc*s require a type signature on all arguments, and a *record likewise requires a type signature on all arguments since it stores them inside of instance variables.

Lookup Table For All Method Data

When that is done, the next step is to build the main lookup table.

The main lookup table matches a type signature to a list of methods that have that signature along with the argument information for each method.

macro build_send_lookup_table
  {%
    table = {} of String => Hash(String, Hash(String, Hash(String, String)))
    @type.methods.reject do |method|
      method.args.any? { |arg| arg.restriction.is_a?(Nop) }
    end.map do |method|
      TypeLookupByLabel[method.args.symbolize]
    end.uniq.each do |restriction|
      @type.methods.each do |method|
        if restriction == TypeLookupByLabel[method.args.symbolize]
          table[restriction] = {
            method.name.stringify => {
              "args" => "#{method.args.map {|arg| "#{arg.name} : #{arg.restriction}" }.join(", ").id}",
              "callargs" => method.args.map { |arg| arg.name }.join(", ")
            }
          }
        end
      end
    end
  %}
  SendLookupTable = {{table.stringify.id}}
end
Enter fullscreen mode Exit fullscreen mode

This table contains all of the key details that are needed to write the rest of the code for implementing #send.

Build The Call Sites To Invoke Methods

The next step is to create the call sites which are used to actually invoke each method. These can be created using either a Proc or a record. There may sometimes be good reasons for using a Proc, but most of the time a record based implementation will be the better choice, so this example will generate only record based call sites.

macro build_callsites
  {% for signature, args in SendLookupTable %}
  {% for method, hsh in args %}
  record Send_{{ method.id }}_{{ signature.id }}, obj : {{ @type.id }}, {{ hsh["args"].id }} do
    def call
      obj.{{ method.id }}({{ hsh["callargs"].id }})
    end
  end 
  {% end %}
  {% end %}
end
Enter fullscreen mode Exit fullscreen mode

This macro iterates the information that was stored in the lookup table and uses it to generate the record wrappers around the method invocations.

Build The Actual Send Methods

The final step is to write all of the #send overloads.

macro build_sends
  {% for signature, args in SendLookupTable %}
  {% for method, hsh in args %}
  def send(method : String | Symbol, {{ hsh["args"].id }})
    Send_{{ method.id }}_{{ hsh["calltype"] }}.new(
      self,
      {{ hsh["callargs"].id }}
    ).call
  end
  {% end %}
  {% end %}
end
Enter fullscreen mode Exit fullscreen mode

And Finally, Run It All!

There is actually one final-final step. One needs a way to add this to an existing class.

Crystal provides an included hook, which will run a macro by that name if one exists when a module is included. This can be used to invoke our macros and make the #send magic possible.

  macro included
    build_type_lookup_table
    build_send_lookup_table
    build_callsites
    build_sends
  end
Enter fullscreen mode Exit fullscreen mode

And How Does It Perform?

For testing, let's define a method, #i_do_not_do_anything, like this:

def i_do_not_do_anything(x : String? = nil)
end
Enter fullscreen mode Exit fullscreen mode

Then we will test both a simple method call that does something, and one that does nothing, and see what we get:

Benchmark.ips do |ips|
  count_a = 0_u64
  count_b = 0_u64
  ips.report("direct - addition") { foo.a(count_a); count_a += 1}
  ips.report("generic send() implementation - addition") { foo.send("a",count_b); count_b += 1 }
  ips.report("direct - i_do_not_do_anything") { foo.i_do_not_do_anything(nil) }
  ips.report("generic send() implementation - i_do_not_do_anything") { foo.send("i_do_not_do_anything",nil) }
end
Enter fullscreen mode Exit fullscreen mode

The results show that there is really no difference at all, when compiled in release mode (--release), between direct method invocation, and methods called through this #send implementation:

                                   direct - addition 514.82M (  1.94ns) (± 1.45%)  0.0B/op   1.80x slower
            generic send() implementation - addition 514.93M (  1.94ns) (± 1.54%)  0.0B/op   1.80x slower
                       direct - i_do_not_do_anything 927.06M (  1.08ns) (± 1.46%)  0.0B/op        fastest
generic send() implementation - i_do_not_do_anything 926.81M (  1.08ns) (± 1.41%)  0.0B/op   1.00x slower
Enter fullscreen mode Exit fullscreen mode

Limitations

It needs to be pointed out that there are extensive limitations in this implementation. It is naive and overlooks many of the complexities of Crystal's type system, method definitions, and method overloading capabilities, so there is a lot of code that the implementation in this article can not call correctly.

Onward...

Use of #send is probably the wrong choice for most Crystal code, but there are times and places where that capability is incredibly useful. If this whets your appetite for something better, a more robust implementation of these same concepts is available on GitHub and as a shard, from here: https://github.com/wyhaines/Send.cr. As of when this article was written, that implementation was itself incomplete, but it has far fewer limitations, and is probably robust enough to be used so long as one is cognizant of the documented limitations.

Crystal macros are very powerful. Despite implementing a decent chunk of the functionality of a usable #send method in just a handful of lines, these examples barely scratch the surface of what is possible with judicious use of macros.

If you want to read more about them, start with the official Crystal documentation. In the future I may explore other aspects of Ruby metaprogramming, and how to achieve equivalent or near-equivalent behaviors in Crystal.

Discussion (0)