The request log says the job was enqueued. The customer’s CRM record still has not changed.
Enqueued SyncCustomerJob (Job ID: 7f9...) to SolidQueue(crm_sync) The queue dashboard shows the job sitting in Solid Queue’s ready state. The error tracker has nothing from SyncCustomerJob, and its retry count remains zero.
Someone adds retry_on Net::ReadTimeout and deploys again. Nothing changes. The team is changing what should happen after a failed attempt without knowing whether an attempt began.
perform_later creates a job
Consider the call that produced that log:
class SyncCustomerJob < ApplicationJob
queue_as :crm_sync
def perform(account_id)
account = Account.find(account_id)
Crm::CustomerSync.call(account)
end
end
job = SyncCustomerJob.perform_later(account.id)It is tempting to read this as a delayed method call: Rails remembers the current call to perform, moves it out of the request, and resumes it when a worker is free.
Active Job does not suspend the current stack frame, ship its local state to another process, and resume it there. Active Job does something more constrained.
In Rails 8.1.3, the class method perform_later first creates a SyncCustomerJob instance. Initialization stores the arguments, assigns a UUID to job_id, and resolves values such as the queue name and priority. It then calls enqueue on that instance.1
At this point, the job is still a Ruby object in the process handling the request:
job.class # SyncCustomerJob
job.arguments # [42]
job.job_id # "7f9..."
job.queue_name # "crm_sync"
job.executions # 0Outside transaction-aware deferral, perform_later returns the job instance when enqueueing succeeds and false when an enqueue callback halts the chain or Active Job handles ActiveJob::EnqueueError. Inside a transaction whose enqueue is deferred until commit, it returns the job instance before that adapter handoff.
Solid Queue’s insertion failures raise a separate SolidQueue::Job::EnqueueError, so they escape the call instead.2
Its perform method has not run. Rails has constructed a description of work and given that description an application-level identity.
perform_now takes the other path. It creates or accepts the job instance and executes it in the current process. perform_later passes its job instance toward a queue adapter instead.3
That difference is larger than a scheduling option. With a production queue adapter, the worker may run later in another process. The current call stack, open files, database connections, and local variables do not travel with it. Instead, the worker receives serialized job data.
That includes state your application makes feel global. If perform depends on Current.account, a client memoized in a service object, or an open file handle from the request, the worker will not inherit it. Rails preserves the locale and time zone by explicitly writing them into the job payload. Anything else must be represented as data the worker can reconstruct.
Active Job stops at the adapter boundary
The instance method enqueue applies options from calls such as set(queue: …) or set(wait: …), initially marks the enqueue as unsuccessful, and runs the enqueue callback chain. Inside that chain, Rails calls one of the two adapter methods:
if scheduled_at
queue_adapter.enqueue_at(self, scheduled_at.to_f)
else
queue_adapter.enqueue(self)
endThe abstract adapter defines those methods by raising NotImplementedError. An adapter must implement the contract or fail loudly.4
Application code constructs one kind of job object, but adapter choice changes the guarantee behind it. Solid Queue persists database rows. The async adapter keeps outstanding work in the process memory, so a restart loses it. Both implement the same Active Job methods, but they don't provide the same durability.5 The backend defines what acceptance, storage, scheduling, and reservation mean after the handoff.
The adapter turns the job into stored work
When an adapter needs a portable representation, Active Job can serialize the job into a hash like this:
{
"job_class" => "SyncCustomerJob",
"job_id" => "7f9...",
"provider_job_id" => nil,
"queue_name" => "crm_sync",
"priority" => 0,
"arguments" => [42],
"executions" => 0,
"exception_executions" => {},
"locale" => "en",
"timezone" => "UTC",
"enqueued_at" => "2026-09-03T09:00:00.000000000Z",
"scheduled_at" => "2026-09-03T09:00:00.000000000Z"
}ActiveJob::Core#serialize supplies this shape. For an immediate enqueue, scheduled_at can still be nil before the adapter runs. Solid Queue sets it to the current time before serializing the job for storage, as shown here. It assigns provider_job_id only after the row is created, so that field is still nil inside the stored payload.6
Solid Queue stores the serialization in a solid_queue_jobs record and prepares an execution record according to the scheduling and concurrency rules. A job that can run now becomes available through the ready-execution path. Workers claim ready executions only from queues that match their queue configuration.7
When a claimed execution runs, Solid Queue merges its database job ID back into the serialized data as provider_job_id before handing the payload to Active Job.
I would treat queue names as deployment configuration, not decoration on a job class. Adding queue_as :crm_sync changes the work that production must consume. A code review that notices the new queue but doesn't check the worker configuration has reviewed only half the change.
A scheduled job may be durably stored but not eligible yet. A ready job may wait behind higher-priority work. A claimed job may belong to a worker that is now unhealthy. If your queue dashboard says only “enqueued,” find out which of those states it has compressed into one label.
A worker creates another job instance
Once a worker claims the work, the backend gives the stored job data back to Active Job. ActiveJob::Execution.execute calls the job class’s deserialize method, which creates an instance and restores metadata such as its IDs, queue, priority, execution count, locale, time zone, and serialized arguments. It then calls perform_now on that instance.
The order inside perform_now is easy to get wrong when reading a job log. Rails increments executions first, then deserializes the arguments, then enters _perform_job, where perform_start.active_job, the perform callbacks, and your perform method run.
stored payload
-> Active Job restores SyncCustomerJob metadata
-> executions increments
-> arguments deserialize
-> perform_start.active_job
-> perform callbacks run
-> perform(42) runs
-> Crm::CustomerSync.call performs the side effectAn unhandled deserialization failure escapes before perform_start.active_job, and the outer perform.active_job instrumentation records the exception. If retry_on handles that failure, retry_job re-enqueues the job after executions has been incremented. The serialized retry therefore climbs toward its attempt limit even though perform has never run. If you see a perform event without a perform-start event, a worker did claim the job. The failure happened before application code began.
The worker’s SyncCustomerJob is not the object returned to the request process. It is reconstructed from stored data, possibly on another machine and after a deploy. Even when both objects carry the same job_id, they exist in different Ruby executions.
I read perform_later as a message boundary. The request says which job class should run, on which queue, with which portable arguments, and with which execution metadata. The worker later interprets that message using the code and application state available to it then.
Argument semantics decide what the request can put into that message. Transaction-aware deferral, introduced in Why after_commit Is a Different Kind of Callback, decides when the adapter first sees it, and a future article will examine what that timing does and doesn’t guarantee. Neither turns the enqueue call, queue storage, worker execution, and completed side effect into one event.
Next in The Background Execution Boundary: passing an Active Record object to a job preserves an identifier, not the object or its current attributes.
Subscribe free for the next week’s source-backed trace of what the worker reconstructs from that argument.
Trace the job through states, not log lines
The new retry_on declaration did nothing for SyncCustomerJob. A job that never starts cannot raise the exception that schedules a retry. Before you change the exception policy, establish how far the work traveled.
For the earlier SyncCustomerJob, collect this sequence:
enqueue intent - job class, Active Job ID, arguments, intended queue
adapter handoff - adapter name, enqueue result, provider job ID, enqueue error
backend state - stored, scheduled, ready, claimed, failed, or absent
worker execution - worker identity, queue subscription, start time, attempt number
application outcome - perform completion and the CRM side effect
The two job identifiers connect those layers. Active Job assigns job_id before the backend sees the job. The adapter may assign provider_job_id after it accepts or persists the work. Keep both: the first connects request and worker logs, while the second locates the backend’s record.
Rails already instruments useful parts of that path. enqueue.active_job and enqueue_at.active_job surround the adapter handoff. perform_start.active_job fires before the application’s perform work, and perform.active_job records its completion or exception.8
An event name by itself does not establish success. The enqueue event still fires when before_enqueue halts the callback chain, with aborted: true, and when an adapter raises ActiveJob::EnqueueError, which Active Job stores on job.enqueue_error. Rails’ own subscribers check the callback outcome, the job’s enqueue error, and any exception that escaped the instrumented block.
Rails 8.1 already attaches ActiveJob::StructuredEventSubscriber. Its enqueue event includes the job class, Active Job ID, queue, adapter, abort state, and exception details. The start event carries the job identity, queue, and enqueue time. The completion event adds the adapter, duration, abort state, and any escaped exception. Use those events when your logging stack consumes Rails structured events. If you also need provider_job_id, a small notification subscriber can add it without pretending that the event itself proves acceptance:
%w[
enqueue.active_job
enqueue_at.active_job
perform_start.active_job
perform.active_job
].each do |event_name|
ActiveSupport::Notifications.subscribe(event_name) do |event|
job = event.payload.fetch(:job)
error = event.payload[:exception_object] || job.enqueue_error
Rails.logger.info [
"event=#{event.name}",
"job_class=#{job.class.name}",
"job_id=#{job.job_id}",
"provider_job_id=#{job.provider_job_id || '-'}",
"queue=#{job.queue_name}",
"adapter=#{job.class.queue_adapter_name}",
"aborted=#{event.payload[:aborted] == true}",
"error=#{error&.class&.name || '-'}"
].join(" ")
end
endqueue_adapter_name is public. Calling it on job.class also preserves a job-specific adapter override that ActiveJob::Base.queue_adapter_name would miss. If you need the concrete class rather than its configured name, use event.payload.fetch(:adapter).class.name.
For an enqueue event, treat the handoff as failed when aborted is true or either error field is present. Then confirm backend storage rather than inferring it from the notification.
Keep arguments out of general job telemetry unless you know they're safe. Active Job logs them by default, and its structured subscriber follows the same setting. You can disable that globally with config.active_job.log_arguments = false or on a sensitive job with self.log_arguments = false.9 Customer identifiers, tokens, uploaded data, and message bodies can turn a debugging aid into a data exposure problem.
The notifications cover the Active Job side of the trace. They cannot tell you everything the backend knows. Follow provider_job_id into the backend and inspect whether the job is scheduled, ready, claimed, failed, or absent. Solid Queue also lets you start with the Active Job ID already present in the application log:
stored_job = SolidQueue::Job.find_by(active_job_id: "7f9...")
[stored_job&.status, stored_job&.ready?, stored_job&.claimed?, stored_job&.failed?]The opening job is present and ready on crm_sync. Now inspect the worker deployment:
production:
workers:
- queues: [default]
threads: 5
processes: 2Every production worker polls default. None is willing to claim crm_sync.
The complete trace now reads:
enqueue event present
provider job row present and ready on crm_sync
matching worker absent
perform-start event absent
CRM side effect absentThe fix is to make a worker consume crm_sync or to route the job to a queue workers already consume. Changing retries, rescuing exceptions, or editing perform would leave the actual failure untouched.
Do not generalize the absent perform-start event without checking perform.active_job. If the outer perform event carries a deserialization exception, the worker claimed the job and failed while rebuilding its arguments. If both worker-side events are absent and the provider record remains ready, worker availability and queue selection are more likely suspects.
A perform completion event needs similar care. An exception handled by retry_on does not escape the instrumented perform_now call, so exception_object can be empty even though Active Job scheduled another attempt. The application side effect remains the final evidence.
Each component owns a narrower guarantee than the method name perform_later suggests:
Active Job → constructs a portable job description and reports the adapter handoff
queue adapter and backend → accept, store, and make the work eligible according to backend rules
worker deployment → runs processes configured to claim that queue
job code and downstream system → complete the intended application side effect
When the side effect is missing, walk that list in order and stop at the first guarantee you cannot prove.
Have you seen a Rails job remain enqueued without reaching perform? Reply or comment with the first missing state you found: adapter acceptance, backend storage, queue consumption, worker start, or the final side effect.
ActiveJob::Enqueuing constructs or accepts a job instance, runs enqueue callbacks, and invokes the configured adapter.
The Rails Active Job guide documents the difference between ActiveJob::EnqueueError and Solid Queue’s enqueue error behavior.
ActiveJob::Execution separates immediate execution from adapter-backed execution and shows the worker-side deserialize-and-perform path.
ActiveJob::QueueAdapters::AbstractAdapter defines enqueue and enqueue_at as the adapter boundary.
The Rails Active Job guide also documents that the async backend keeps jobs in RAM and loses outstanding work when the process exits, while Solid Queue persists work in a database.
ActiveJob::Core assigns job metadata and produces the hash passed safely to queue adapters.
The Solid Queue 1.4.0 job model sets scheduled_at, stores active_job.serialize with backend metadata, and assigns provider_job_id after persistence. The Rails Active Job guide describes ready and scheduled executions and worker queue selection.
ActiveJob::Instrumentation emits the enqueue and perform events. Rails’ StructuredEventSubscriber handles aborted enqueues, enqueue errors, and exceptions when producing structured events.
ActiveJob::Logging defines log_arguments, which defaults to true and can be configured globally or per job class.

I've definitely been guilty of tweaking retry_on before actually checking if the worker process was even listening to the right queue...