tim.uru.ski

This is part three in a series of posts about signal handling in Ruby.

Now that we have an implementation of GracefulShutdown, let’s write a basic test to verify it works correctly. The simplest way to test this is with fork. It splits our program in two, then we can then send an interrupt signal from parent to child and assert it exited without error.

describe GracefulShutdown do
  it "exits without error" do
    ruby = fork do
      GracefulShutdown.new.handle_signals do
        sleep 1.0
        raise 'No Interrupt received'
      end
    end

    # Ensure process is running.
    sleep 0.01

    Process.kill('INT', ruby)
    _, status = Process.waitpid2(ruby)

    expect(status.success?).to be true
  end
end

This test is pretty straightforward, but I’ll summarize a few key points.

When fork is called it returns a process ID (or pid), which can be used to track and signal the child process. The block passed to fork is executed only by the child and it exits when the block is complete.

Process.kill is used to send INT, the interrupt signal to be handled by GracefulShutdown. Inside the test block, an exception is raised after a moment. We could have it block indefinitely, but our test suite would then hang instead of giving us a meaningful failure.

The call to Process.waitpid2 lets the test program block until the child is finished. It also returns the pid (ignored by assigning it to _) and a Status object that can be used to check if the child was successful, ie. ended with exit code 0.

So far so good, but there is the matter of calling sleep 0.1 after starting the test process. This is required to ensure the GracefulShutdown block is being executed. But because the startup time of the fork can be influenced by outside factors, it can fail at random. We end up with a “flapping” test. This can break build pipelines and cause other mischief so let’s try to refactor that detail away.

First we’ll rewrite our test code so it has better structure to support the changes.

it "exits without error" do
  test = RubyBlock.new do |helper|
    GracefulShutdown.new.handle_signals do
      helper.wait_for_signal 1.0
      raise 'No Interrupt received'
    end
  end

  test.run_and_send('INT')

  expect(test).to be_successful
end

The fork method is replaced with RubyBlock.new, it accepts the test block and now takes a helper argument.

Instead of the sleep call inside the block we use helper.wait_for_signal. This adds some clarity and also provides a mechanism to notify our parent process that the handler is being executed and actually ready to receive a signal. Lastly we’ll call test.run_and_send, that will run our test and send the signal when the wait_for_signal call is made.

Next we can create the RubyBlock class to represent the test process and provide the successful? helper for our spec.

class RubyBlock
  def initialize(&test_block)
    @test_block = test_block
  end

  def run_and_send(signal)
    # IMPLEMENT ME
  end

  def successful?
    @status && @status.success?
  end
end

To avoid the arbitrary sleep, we’ll instead use the custom signal USR1. Our child process can send it to the parent as a cue that the block is ready. In our parent we can trap(:USR1) and send the test signal.

def run_and_send(signal)
  trap(:USR1) do
    Process.kill(signal, pid)
  end

  pid = fork do
    @test_block.call(self)
  end

  _, @status = Process.waitpid2(pid)
ensure
  trap(:USR1, 'DEFAULT')
end

Note that self is passed into @test_block.call, this is the best way to provide our wait_for_signal helper. We could inject it into the global namespace, but is generally frowned upon, so we’ll avoid it. We could also use instance_exec but the test block will lose scope with the test environment, also not ideal.

We also use an ensure block to restore the default behaviour for USR1 so it doesn’t misfire if there are multiple tests setting traps for it.

The wait_for_signal helper is straightforward, it sends the signal and then sleeps for a moment.

def wait_for_signal(seconds)
  Process.kill('USR1', Process.ppid)
  sleep(seconds)
end

So now we have all the pieces we need to test our signal handler. But there’s one more problem. Signals are great for asychronous communication between processes, but they aren’t completely reliable. We can do one final refactoring of our RubyProcess class to use an IO pipe instead.

The run_and_send method gets refactored like this:

def run_and_send(signal)
  block_rd, @block_wr = IO.pipe

  pid = fork do
    @test_block.call(self)
  end

  block_rd.gets
  Process.kill(signal, pid)
  _, @status = Process.waitpid2(pid)
ensure
  block_rd.closed? or block_rd.close
  @block_wr.closed? or @block_rd.close
end

The signal trap has been replaced with a call to IO.pipe, which returns a pair of read and write IO objects. The write end is stored as @block_wr so the wait_for_signal helper can write to it. The fork portion is unchanged, but next we use block_rd.gets to block the parent process until the child is ready. There is also an ensure section to this method to close the pipe and keep things tidy.

The changes to wait_for_signal are similarly simple, instead of sending USR1 to the parent, it just writes 'READY' into the @block_wr end of the pipe and then flushes it so the parent process will stop blocking.

  def wait_for_signal(seconds)
    @block_wr.puts 'READY'
    @block_wr.flush
    sleep(seconds)
  end

Now our parent and child process can work together without worry if one gets delayed. Here’s the full code all put together. The RubyBlock helper can be moved into a support file to keep the test code focused.

class RubyBlock
  def initialize(&test_block)
    @test_block = test_block
  end

  def wait_for_signal(seconds)
    @block_wr.puts 'READY'
    @block_wr.flush
    sleep(seconds)
  end

  def run_and_send(signal)
    block_rd, @block_wr = IO.pipe

    pid = fork do
      @test_block.call(self)
    end

    block_rd.gets
    Process.kill(signal, pid)
    _, @status = Process.waitpid2(pid)
  ensure
    block_rd.closed? or block_rd.close
    @block_wr.closed? or @block_wr.close
  end

  def successful?
    @status && @status.success?
  end
end

describe GracefulShutdown do
  it "exits without error" do
    test = RubyBlock.new do |helper|
      GracefulShutdown.new.handle_signals do
        helper.wait_for_signal 1.0
        raise 'No Interrupt received'
      end
    end

    test.run_and_send('INT')

    expect(test).to be_successful
  end
end

Bonus Content

If you checkout out the code for GracefulShutdown that was posted to Github with the last post, you may have noticed an alternate way to write these tests that uses popen instead of fork.

Here’s the interesting parts:

class RubyProcess
  WAIT_HELPER = <<-EOS
    def wait_for_signal(seconds)
      # Signal parent we are ready.
      Process.kill('USR1', Process.ppid)

      # Wait for interrupt.
      sleep(seconds)
    end
  EOS

  def initialize(code, *args)
    cmd = ['ruby'] + args
    @io = IO.popen(cmd, 'w+', err: [:child, :out])
    @io.write(WAIT_HELPER)
    @io.write(code)
  end

  def run_and_send(signal)
    usr1_handler = trap(:USR1) do
      Process.kill(signal, @io.pid)
    end

    # The program starts when the write end of the pipe is closed.
    @io.close_write
    _, @status = Process.waitpid2(io.pid)
  ensure
    trap(:USR1, usr1_handler)
  end

  def successful?
    @status.success?
  end

  def output
    @io.read
  end
end

This starts a fresh Ruby process which has the test code fed into it on STDIN, the test code takes the form of a string. Ruby will delay execution of the test code until it reaches the end of the file, so we can control execution until the @io is closed for writing.

The helper is another block of text that defines wait_for_signal at the top level, it gets injected into the input stream in the constructor, ahead of the test code. Because it doesn’t share anything with the parent process, we can’t use the IO.pipe trick as easily, so it still uses the signal system we covered earlier.

In some ways this code is better. It’s a clean slate, so there’s no chance of some test state leaking into the example, it’s also simpler, just a single input into the process. But it forces tests to be written as strings, which isn’t ideal, and the use of signals to synchronize things is also prone to failure. Overall the fork method is superior, even with the added complexity.