Consistent, request-local state

👋🏼 This is a series on concurrency, parallelism and asynchronous programming in Ruby. It’s a deep dive, so it’s divided into 12 main parts:

• Your Ruby programs are always multi-threaded: Part 1
• Your Ruby programs are always multi-threaded: Part 2
  • Consistent, request-local state
• Ruby methods are colorless
• The Thread API: Concurrent, colorless Ruby
• Interrupting Threads: Concurrent, colorless Ruby
• Thread and its MaNy friends: Concurrent, colorless Ruby
• Fibers: Concurrent, colorless Ruby
• Processes, Ractors and alternative runtimes: Parallel Ruby
• Scaling concurrency: Streaming Ruby
• Abstracted, concurrent Ruby
• Closing thoughts, kicking the tires and tangents
• How I dive into CRuby concurrency

You’re reading “Consistent, request-local state”. I’ll update the links as each part is released, and include these links in each post.

Rather than a full entry in the series, this is a brief-ish¹ concurrency mini - a follow up to Your Ruby programs are always multi-threaded: Part 2.

Recap

In Sharing state with fibers, we walked through a scenario where you had fiber-local state using Thread.current[], and it was not available inside nested Fibers.

# This is #<User id=123>
Thread.current[:app_user]
Async do # equivalent to Fiber.new
  # This is nil
  Thread.current[:app_user]
  params[:files].each do |file|
    Async { # Fiber.new
      # Still nil
      Thread.current[:app_user]
      # Upload to S3 fails...
    }
  end
end

We had fanned out user uploads to take advantage of parallelized IO. The problem was we tried to use fiber-local state and that gets reset in new Fibers.

To solve it, we tried true thread-local state with thread_variable_get and thread_variable_set.

# This is #<User id=123>
Thread.current.thread_variable_get(:app_user)
Async do
  # Still #<User id=123>
 Thread.current.thread_variable_get(:app_user)
  params[:files].each do |file|
    Async {
      # Still #<User id=123>!
      Thread.current.thread_variable_get(:app_user)
      # Upload to S3...
    }
  end
end

On a regular threaded server, that’s ok. It’s exactly how CurrentAttributes would function by default:

class AppContext < ActiveSupport::CurrentAttributes
  attribute :user, :locale

  def user=(user)
    super
    self.locale = user.locale
  end
end

# This is #<User id=123>
AppContext.user
Async do
  # Still #<User id=123>
  AppContext.user
  params[:files].each do |file|
    Async {
      # Still #<User id=123>!
      AppContext.user
      # Upload to S3...
    }
  end
end

That’s because CurrentAttributes works off of an isolation_level. It internally uses ActiveSupport::IsolatedExecutionContext, which is an abstraction on top of Thread/Fiber-local state. It has two possible isolation_levels:

• :thread - This is the default, and works exactly like our code using thread_variable_get²
• :fiber - This is the other available option, and works exactly like our code using Thread.current[]³

ActiveSupport::IsolatedExecutionContext.isolation_level = :thread # or :fiber

The reason this abstraction matters is because of the introduction of Fiber-based servers.

Careful with the Falcons

If you look into the async ecosystem, pretty quickly you’ll find a full-featured web server called falcon.

Falcon is a multi-process, multi-fiber rack-compatible HTTP server built on top of async, async-container and async-http. Each request is executed within a lightweight fiber and can block on up-stream requests without stalling the entire server process. Falcon supports HTTP/1 and HTTP/2 natively.

falcon is Fiber-based, meaning each request is run in a new Fiber, rather than a new Thread. Effectively, as it accepts each socket connection it hands it off to a new Fiber:

Async do # Fiber.new
  socket.accept do
    # Instead of Thread.new or a Thread pool
    Async {} # Fiber.new
  end
end

By default, falcon will set the IsolatedExecutionContext.isolation_level for you to use Fibers:

ActiveSupport::IsolatedExecutionContext.isolation_level = :fiber

Which means it localizes its state at the Fiber level. We’ve got our original problem again:

# This is #<User id=123>
AppContext.user
Async do
  # This is nil
  AppContext.user
  params[:files].each do |file|
    Async {
      # Still nil
      AppContext.user
      # Upload to S3 fails...
    }
  end
end

What would happen if you set the isolation_level back to :thread?

ActiveSupportIsolatedExecutionContext.isolation_level = :thread

class ImagesController
  before_action :set_user
	
  def create
    Async do
      # May be a user from a different fiber!
      # They all live in the same Thread, so they share
      #   thread-local state
      AppContext.app_user
      params[:files].each do |file|
        Async { S3Upload.new(file).call }
      end
    end
  end
end

Bad things. We start running into issues with shared global state again, because all of our fibers share the same Thread! We need to be Fiber-local when running on a Fiber-based server.

If Thread-locals are not safe on a Fiber-based server, how can we safely share this state?

Resetting context

One less than ideal option is to just reset the state in each nested Fiber:

# This is #<User id=123>
user = AppContext.user
Async do
  AppContext.user = user
  # This is #<User id=123>
  AppContext.user
  params[:files].each do |file|
    Async {
      AppContext.user = user
      # This is #<User id=123>
      AppContext.user
      # Upload to S3...
    }
  end
end

You don’t have to set it on every level - just the level that actually needs it. But you probably want to set it on every level to not surprise yourself later.

This works, but it’s manual and clunky.

Fiber storage

There’s a new option available as of Ruby 3.2.

It’s a new kind of “local”, called Fiber storage⁴, and it was designed to help with precisely this type of issue:

Fiber[:app_user] = user

# This is #<User id=123>
Fiber[:app_user]
Async do
  # This is #<User id=123>
  Fiber[:app_user]
  params[:files].each do |file|
    Async {
      # This is #<User id=123>
      Fiber[:app_user]
      # Upload to S3...
    }
  end
end

Fiber storage is a mechanism for inheriting state from a Fiber to any Ractors/Threads/Fibers started from that scope.

The best term for this type of storage is “request-local”. The definition of “request” i’m using here is very loose - it just means the context of some particular slice of execution. That might mean a web page request, a background job run, a cron job, etc.

So we could create a new AppContext using this approach:

class AppContext
  class << self
    def user
      Fiber[:app_user]
    end

    def user=(user)
      Fiber[:app_user] = user
      self.locale = user.locale
    end

    def locale
      Fiber[:app_locale]
    end

    def locale=(locale)
      Fiber[:app_locale] = locale
    end
  end
end

With this new AppContext, things are working again!

Async do
  # This is #<User id=123>
  AppContext.user
  params[:files].each do |file|
    Async {
      # This is #<User id=123>
      AppContext.user
      # Upload to S3...
    }
  end
end

Even more, this works for Threads too:

Thread.new do
  # This is #<User id=123>
  AppContext.user
  params[:files].each do |file|
    Thread.new {
      # This is #<User id=123>
      AppContext.user
      # Upload to S3...
    }
  end
end

And Ractors:

Ractor.new do
  # This is #<User id=123>
  AppContext.user
  Thread.new do
    # This is #<User id=123>
    AppContext.user
  end.join
end

📝 If it works for Ractors, Threads and Fibers, why call it “Fiber storage”?

Fibers are the innermost level of concurrency in Ruby (Process -> Ractor -> Thread -> Fiber). Since all code operates within a Fiber scope, it makes sense to have it be the storage layer for local state.

Every time you create a new Ractor/Thread, a new Fiber is created within it. Which means the thing actually inheriting the Fiber storage is the Fiber that exists within that new context, not the Ractor/Thread itself.

An ideal future state of libraries is that everything handling “request” type state would move to using Fiber storage (CurrentAttributes, RequestStore, and friends).

Caveats

Like with any new approach in an existing, robust ecosystem, there are some caveats to consider.

It requires framework buy-in

Every request needs to be wrapped in a new Fiber with fresh storage, so it doesn’t accidentally inherit cross-request storage:

# In a library like Puma/Sidekiq/Pitchfork
def new_request
  Fiber.new(storage: nil) do
    # fresh storage for this request
  end
end

Falcon already handles this for you. But no other framework does (yet). If you aren’t using Falcon, this may not be a viable option for you yet.

Each layer inherits a copy

Each layer copies the keys and values from the current Fiber storage scope into a new hash. You can’t override entries from the parent scope, so overrides only impact lower levels.

Fiber[:value] = 1
Fiber.new do
  Fiber[:value] = 2
end
Fiber[:value] # => 1

This is intentional, but just something to keep in mind.

Some things aren’t meant to be shared

As you fan out there are things you likely want to share, like the current user. But there are things you don’t want to, or can’t - like database connections or file handles.

That means existing solutions that try to share concepts (like CurrentAttributes and IsolatedExecutionContext) will need to split their approach. For a concept like CurrentAttributes you want inherited Fiber storage. For a database connection, you want things fiber-local so each Fiber has its own connection.

Takeaways

• Use an abstraction, like CurrentAttributes. Over time, these abstractions should catch up with available options like Fiber storage and you’ll reap the benefits
• If you’re using Falcon, Fiber storage is a great option for sharing state across child Fibers/Threads/Ractors
• If you’re not using Falcon, there’s still some work for the community to catch up with this new option, so use it carefully or avoid it for now

Onward!

Ruby 3+ has introduced interesting new options for concurrency like the FiberScheduler/Fiber storage and thanks to many community efforts, things are starting to catch up. But there’s still work to be done and new abstractions needed to handle things seamlessly.

Thanks to Samuel Williams⁵ (@ioquatix, author of Async, Falcon, and the FiberScheduler) for suggesting the clarification and reviewing this post!

Ok, now we’ll be heading over to “Ruby methods are colorless”. More soon 👋🏼.