In Peephole optimizations: adding opt_respond_to to the Ruby VM, part 4, we dug deep. We found the connection between prism compilation and the specialization we need for our new bytecode, called “peephole optimization”. We learned how to debug and step through C code in the Ruby runtime, and we added some logic for matching the “pattern” of the instruction we want to change.

Now that we know where the specialization needs to go and how to match what needs to be specialized - what do we actually replace it with? How do we get the virtual machine to recognize opt_respond_to?

Pattern matching bytecode instructions

opt_ary_freeze has been a great learning tool - let’s see what it teaches us about adding a new instruction name.

Here’s a refresher on how iseq_peephole_optimize matches on newarray, and then replaces it with opt_ary_freeze:

if (
  IS_INSN_ID(iobj, newarray)
) {
  LINK_ELEMENT *next = iobj->link.next;
  if (
    IS_INSN(next) &&
    IS_INSN_ID(next, send)
  ) {
    //... more if statements
    iobj->insn_id = BIN(opt_ary_freeze);
  • it first checks if the instruction id of iobj is newarray
  • it grabs the next element and checks if its an instruction, then checks if the instruction is send (it also checks if the method id is “freeze”, not shown above)
  • if those checks match, it replaces the instruction id with opt_ary_freeze

That’s pretty reasonable to follow. But how do IS_INSN and IS_INSN_ID work? What is BIN? What type is opt_ary_freeze - where is it defined? How do we add new instructions ourselves?

Macros and enums

BIN, IS_INSN and IS_INSN_ID are all C macros that revolve around interacting with virtual machine instructions.

Macros in C get embedded directly into your code in a preprocessing step before being compiled, so you can write things that look pretty odd compared to a typical C-like syntax. Here’s the definition for BIN:

#define BIN(n) YARVINSN_##n

📝 BIN probably stands for “Binary INstruction”

That ## is kind of like string interpolation, but the result is a static part of your actual code. This means that anywhere BIN is called, it’s kind of like saying YARVINSN_#{n} in Ruby. So this code:

iobj->insn_id = BIN(opt_ary_freeze);

Gets transformed into this, right before the program is compiled:

iobj->insn_id = YARVINSN_opt_ary_freeze;

Here’s the definition for IS_INSN_ID:

#define IS_INSN_ID(iobj, insn) (INSN_OF(iobj) == BIN(insn))

Based on our understanding of macros and BIN, so far, it gets transformed into:

#define IS_INSN_ID(iobj, insn) (INSN_OF(iobj) == YARVINSN_##insn)

Here’s the definition for INSN_OF, it just casts insn to an INSN type, and accesses its instruction id:

#define INSN_OF(insn) \
  (((INSN*)(insn))->insn_id)

That means the expanded version of IS_INSN_ID is:

#define IS_INSN_ID(iobj, insn) \
  (((INSN*)(insn))->insn_id == YARVINSN_##insn)

Here’s the definition for IS_INSN:

#define IS_INSN(link) ((link)->type == ISEQ_ELEMENT_INSN)

If we combined all of it together, and manually inline it, here’s what our original pattern matching code looks like:

if (
  (((INSN*)(iobj))->insn_id == YARVINSN_newarray)
) {
  LINK_ELEMENT *next = iobj->link.next;
  if (
    (next)->type == ISEQ_ELEMENT_INSN &&
    (((INSN*)(next))->insn_id == YARVINSN_send)
  ) {
    //... more if statements
    iobj->insn_id = YARVINSN_opt_ary_freeze;

I’m glad CRuby adds those macros… this expanded code is a lot less readable.

Why did I expand it and make that original code less clear? I wanted to think through what the instructions really look like at runtime, and why. The reason I can infer, is that all of these macros let you focus on a syntax that looks just like our VM instructions, while making sure there are no name collisions behind the scenes.

Ok, I still haven’t shown where the instruction comes from. Here’s the file you can actually find an enum containing the entire list of vm instructions:

// insns.inc
enum ruby_vminsn_type {
  BIN(nop),
  BIN(getlocal),
  //...
  BIN(opt_ary_freeze),
  //...
}

// or, expanded by the preprocessor!
enum ruby_vminsn_type {
  YARVINSN_nop,
  YARVINSN_getlocal,
  //...
  YARVINSN_opt_ary_freeze,
  //...
}

insns.inc gets included anywhere we need instruction checks, like in compile.c. These enum values are globally available anywhere this file is included. Thanks to BIN prepending all of their names with YARVINSN_, we can use them in a convenient syntax without having any collisions.

So if I search the CRuby repo for insns.inc, where can I find it? Hmmm, I can’t 🤔. insns.inc is a generated file! I can only see it locally, after compiling the entire project. Where does that file get generated from?

A virtual machine DSL

While insns.inc tells us the name of each instruction available, the file it is generated from defines every instruction available in the Ruby virtual machine, and how it should respond to that instruction. It’s called insns.def. The file looks a lot like C, but it’s actually a kind of DSL.

It lets you define a simplified set of information for the instruction. That simplified format is then compiled into a more comprehensive, C compatible version.

It’s compilers all the way down… 😵‍💫

The top of the file defines the format. I don’t fully understand it, but let’s walk through it:

/*
DEFINE_INSN
instruction_name
(type operand, type operand, ..)
(pop_values, ..)
(return values ..)
// attr type name contents..
{
  .. // insn body
}
*/

An instructions consists of:

  • A name ✅
  • Operands - like our “call data”. I’m not sure what other types of operands are typically used for, but I know opt_ary_freezs puts the frozen array here as well
  • Values to pop off the virtual machine stack so we can operate on them. This should be values that we’ve seen pushed onto the stack in previous instructions
  • A return value
  • (I don’t fully understand the value of attr type but it seems to influence what code gets generated by the instruction definition)
  • A C-compatible body

That’s a lot, baked into a small interface. Let’s look at a very simple example. Here’s one of the simplest instructions available, putnil:

DEFINE_INSN
putnil
()
()
(VALUE val)
{
    val = Qnil;
}

Looks… pointless? Theputnil instruction takes no arguments, and has a return value of val. The only thing the code block does is set val equal to Qnil, which is a special value in CRuby representing Ruby’s nil. What does that accomplish?

📝 VALUE is a special type in CRuby that points at a Ruby object, usually located on the heap. When you see VALUE, this often means we’re looking at a value you’d use in a Ruby program.

This file is compiled into regular C code, and the context of this simple instruction becomes clearer:

// vm.inc
/* insn putnil()()(val) */
INSN_ENTRY(putnil)
{
  //...
  VALUE val;
  //...
#   define NAME_OF_CURRENT_INSN putnil
#   line 331 "../insns.def"
{
  val = Qnil;
}
  //...
  INC_SP(INSN_ATTR(sp_inc));
  TOPN(0) = val;
  //...
}
  • The return value VALUE val is declared
  • It’s set to val = Qnil, the instruction we saw in insns.def
  • INC_SP is called, which I believe “increments” the “stack pointer”, giving us extra space on the stack to push onto?
  • TOPN(0) = val sets val to the top of the stack

I think I’ll dig more into that next time. But let’s get back to the task at hand - it’s time to try and get our respond_to? call replaced with opt_respond_to!

Adding to the DSL

It took me a bit of banging my head against a wall, but here is the working instruction and specialization, in a basic form:

// insns.def
DEFINE_INSN
opt_respond_to
(CALL_DATA cd)
(VALUE recv, VALUE mid)
(VALUE val)
{
    val = vm_opt_respond_to(recv, mid);
    CALL_SIMPLE_METHOD();
}

// compile.c
if (IS_INSN_ID(iobj, send)) {
  const struct rb_callinfo *ci = (struct rb_callinfo *)OPERAND_AT(iobj, 0);
  const rb_iseq_t *blockiseq = (rb_iseq_t *)OPERAND_AT(iobj, 1);

  if (vm_ci_simple(ci) && vm_ci_argc(ci) == 1 && blockiseq == NULL && vm_ci_mid(ci) == idRespond_to) {
      iobj->insn_id = BIN(opt_respond_to);
      iobj->operand_size = 1;
      iobj->operands = compile_data_calloc2(iseq, iobj->operand_size, sizeof(VALUE));
      iobj->operands[0] = (VALUE)ci;
  }
}

If we dump the instructions, we finally see our new instruction opt_respond_to. It’s not really doing anything yet, but it’s there!

puts "Did you know you can write to $stdout?" if $stdout.respond_to?(:write)

# > RUNOPT0=--dump=insns make run
# == disasm: #<ISeq:<main>./test.rb:1 (1,0)-(1,76)>
# 0000 getglobal                :$stdout                  (   1)[Li]
# 0002 putobject                :write
# 0004 opt_respond_to           <calldata!mid:respond_to?, argc:1, ARGS_SIMPLE>
# 0006 branchunless             14
# 0008 putself
# 0009 putchilledstring         "Did you know you can write to $stdout?"
# 0011 opt_send_without_block   <calldata!mid:puts, argc:1, FCALL|ARGS_SIMPLE>
# 0013 leave
# 0014 putnil
# 0015 leave

Sorry to just dump the code here, and split. We’ll dig into it more, and expand on it next time! There is lots more to do, and to explain, but i’m excited about this milestone! See you next time! 👋🏼

PS - since something is working now, i’ve pushed up my basic code so far, here.