šš¼ This is a series on concurrency, parallelism and asynchronous programming in Ruby. Itās a deep dive, so itās divided into 12 main parts:
- Your Ruby programs are always multi-threaded: Part 1
- Your Ruby programs are always multi-threaded: Part 2
- Consistent, request-local state
- Ruby methods are colorless
- The Thread API: Concurrent, colorless Ruby
- Interrupting Threads: Concurrent, colorless Ruby
- Thread and its MaNy friends: Concurrent, colorless Ruby
- Fibers: Concurrent, colorless Ruby
- Processes, Ractors and alternative runtimes: Parallel Ruby
- Scaling concurrency: Streaming Ruby
- Abstracted, concurrent Ruby
- Closing thoughts, kicking the tires and tangents
- How I dive into CRuby concurrency
Youāre reading āConsistent, request-local stateā. Iāll update the links as each part is released, and include these links in each post.
Rather than a full entry in the series, this is a brief-ish1 concurrency mini - a follow up to Your Ruby programs are always multi-threaded: Part 2.
Recap
In Sharing state with fibers, we walked through a scenario where you had fiber-local state using Thread.current[]
, and it was not available inside nested Fibers.
# This is #<User id=123>
Thread.current[:app_user]
Async do # equivalent to Fiber.new
# This is nil
Thread.current[:app_user]
params[:files].each do |file|
Async { # Fiber.new
# Still nil
Thread.current[:app_user]
# Upload to S3 fails...
}
end
end
We had fanned out user uploads to take advantage of parallelized IO. The problem was we tried to use fiber-local state and that gets reset in new Fibers.
To solve it, we tried true thread-local state with thread_variable_get
and thread_variable_set
.
# This is #<User id=123>
Thread.current.thread_variable_get(:app_user)
Async do
# Still #<User id=123>
Thread.current.thread_variable_get(:app_user)
params[:files].each do |file|
Async {
# Still #<User id=123>!
Thread.current.thread_variable_get(:app_user)
# Upload to S3...
}
end
end
On a regular threaded server, thatās ok. Itās exactly how CurrentAttributes
would function by default:
class AppContext < ActiveSupport::CurrentAttributes
attribute :user, :locale
def user=(user)
super
self.locale = user.locale
end
end
# This is #<User id=123>
AppContext.user
Async do
# Still #<User id=123>
AppContext.user
params[:files].each do |file|
Async {
# Still #<User id=123>!
AppContext.user
# Upload to S3...
}
end
end
Thatās because CurrentAttributes
works off of an isolation_level
. It internally uses ActiveSupport::IsolatedExecutionContext
, which is an abstraction on top of Thread/Fiber-local state. It has two possible isolation_level
s:
:thread
- This is the default, and works exactly like our code usingthread_variable_get
2:fiber
- This is the other available option, and works exactly like our code usingThread.current[]
3
ActiveSupport::IsolatedExecutionContext.isolation_level = :thread # or :fiber
The reason this abstraction matters is because of the introduction of Fiber-based servers.
Careful with the Falcons
If you look into the async
ecosystem, pretty quickly youāll find a full-featured web server called falcon
.
Falcon is a multi-process, multi-fiber rack-compatible HTTP server built on top of async, async-container and async-http. Each request is executed within a lightweight fiber and can block on up-stream requests without stalling the entire server process. Falcon supports HTTP/1 and HTTP/2 natively.
falcon
is Fiber-based, meaning each request is run in a new Fiber, rather than a new Thread. Effectively, as it accepts each socket connection it hands it off to a new Fiber:
Async do # Fiber.new
socket.accept do
# Instead of Thread.new or a Thread pool
Async {} # Fiber.new
end
end
By default, falcon
will set the IsolatedExecutionContext.isolation_level
for you to use Fibers:
ActiveSupport::IsolatedExecutionContext.isolation_level = :fiber
Which means it localizes its state at the Fiber level. Weāve got our original problem again:
# This is #<User id=123>
AppContext.user
Async do
# This is nil
AppContext.user
params[:files].each do |file|
Async {
# Still nil
AppContext.user
# Upload to S3 fails...
}
end
end
What would happen if you set the isolation_level
back to :thread
?
ActiveSupportIsolatedExecutionContext.isolation_level = :thread
class ImagesController
before_action :set_user
def create
Async do
# May be a user from a different fiber!
# They all live in the same Thread, so they share
# thread-local state
AppContext.app_user
params[:files].each do |file|
Async { S3Upload.new(file).call }
end
end
end
end
![](https://cdn.uploads.micro.blog/98548/2024/ruby-is-all-threads-consistent-state.drawio.png)
Bad things. We start running into issues with shared global state again, because all of our fibers share the same Thread! We need to be Fiber-local when running on a Fiber-based server.
If Thread-locals are not safe on a Fiber-based server, how can we safely share this state?
Resetting context
One less than ideal option is to just reset the state in each nested Fiber:
# This is #<User id=123>
user = AppContext.user
Async do
AppContext.user = user
# This is #<User id=123>
AppContext.user
params[:files].each do |file|
Async {
AppContext.user = user
# This is #<User id=123>
AppContext.user
# Upload to S3...
}
end
end
You donāt have to set it on every level - just the level that actually needs it. But you probably want to set it on every level to not surprise yourself later.
This works, but it’s manual and clunky.
Fiber storage
Thereās a new option available as of Ruby 3.2.
Itās a new kind of ālocalā, called Fiber storage4, and it was designed to help with precisely this type of issue:
Fiber[:app_user] = user
# This is #<User id=123>
Fiber[:app_user]
Async do
# This is #<User id=123>
Fiber[:app_user]
params[:files].each do |file|
Async {
# This is #<User id=123>
Fiber[:app_user]
# Upload to S3...
}
end
end
Fiber storage is a mechanism for inheriting state from a Fiber to any Ractors/Threads/Fibers started from that scope.
The best term for this type of storage is ārequest-localā. The definition of ārequestā iām using here is very loose - it just means the context of some particular slice of execution. That might mean a web page request, a background job run, a cron job, etc.
So we could create a new AppContext
using this approach:
class AppContext
class << self
def user
Fiber[:app_user]
end
def user=(user)
Fiber[:app_user] = user
self.locale = user.locale
end
def locale
Fiber[:app_locale]
end
def locale=(locale)
Fiber[:app_locale] = locale
end
end
end
With this new AppContext
, things are working again!
Async do
# This is #<User id=123>
AppContext.user
params[:files].each do |file|
Async {
# This is #<User id=123>
AppContext.user
# Upload to S3...
}
end
end
Even more, this works for Threads too:
Thread.new do
# This is #<User id=123>
AppContext.user
params[:files].each do |file|
Thread.new {
# This is #<User id=123>
AppContext.user
# Upload to S3...
}
end
end
And Ractors:
Ractor.new do
# This is #<User id=123>
AppContext.user
Thread.new do
# This is #<User id=123>
AppContext.user
end.join
end
![](https://cdn.uploads.micro.blog/98548/2024/ruby-is-all-threads-inherited-fiber-storage.drawio.png)
š If it works for Ractors, Threads and Fibers, why call it āFiber storageā?
Fibers are the innermost level of concurrency in Ruby (Process -> Ractor -> Thread -> Fiber). Since all code operates within a Fiber scope, it makes sense to have it be the storage layer for local state.
Every time you create a new Ractor/Thread, a new Fiber is created within it. Which means the thing actually inheriting the Fiber storage is the Fiber that exists within that new context, not the Ractor/Thread itself.
An ideal future state of libraries is that everything handling ārequestā type state would move to using Fiber storage (CurrentAttributes, RequestStore, and friends).
Caveats
Like with any new approach in an existing, robust ecosystem, there are some caveats to consider.
It requires framework buy-in
Every request needs to be wrapped in a new Fiber with fresh storage, so it doesnāt accidentally inherit cross-request storage:
# In a library like Puma/Sidekiq/Pitchfork
def new_request
Fiber.new(storage: nil) do
# fresh storage for this request
end
end
Falcon already handles this for you. But no other framework does (yet). If you arenāt using Falcon, this may not be a viable option for you yet.
Each layer inherits a copy
Each layer copies the keys and values from the current Fiber storage scope into a new hash. You canāt override entries from the parent scope, so overrides only impact lower levels.
Fiber[:value] = 1
Fiber.new do
Fiber[:value] = 2
end
Fiber[:value] # => 1
This is intentional, but just something to keep in mind.
Some things arenāt meant to be shared
As you fan out there are things you likely want to share, like the current user. But there are things you donāt want to, or canāt - like database connections or file handles.
That means existing solutions that try to share concepts (like CurrentAttributes
and IsolatedExecutionContext
) will need to split their approach. For a concept like CurrentAttributes
you want inherited Fiber storage. For a database connection, you want things fiber-local so each Fiber has its own connection.
Takeaways
- Use an abstraction, like
CurrentAttributes
. Over time, these abstractions should catch up with available options like Fiber storage and youāll reap the benefits - If youāre using Falcon, Fiber storage is a great option for sharing state across child Fibers/Threads/Ractors
- If youāre not using Falcon, thereās still some work for the community to catch up with this new option, so use it carefully or avoid it for now
Onward!
Ruby 3+ has introduced interesting new options for concurrency like the FiberScheduler/Fiber storage and thanks to many community efforts, things are starting to catch up. But thereās still work to be done and new abstractions needed to handle things seamlessly.
Thanks to Samuel Williams5 (@ioquatix, author of Async, Falcon, and the FiberScheduler) for suggesting the clarification and reviewing this post!
Ok, now weāll be heading over to āRuby methods are colorlessā. More soon šš¼.
-
Some readers may be thinking āoh thank god, finally a brief oneā š ↩︎
-
It works the same but itās not actually using that internally. It monkey-patches Thread and adds a new
attr_accessor
. If someone cares, I could talk about it more! ↩︎ -
Also not actually using that internally, same as with
:thread
. It monkey-patches Fiber and adds a newattr_accessor
. If someone cares, I could talk about it more as well! ↩︎ -
Thereās a bit more context in the original proposal https://bugs.ruby-lang.org/issues/19078 and in the docs for creating new Fibers https://docs.ruby-lang.org/en/master/Fiber.html#method-c-new ↩︎
-
Aka Mr Async, aka The Fiber Fella ↩︎