Workers

Workers are processes that claim jobs from queues and execute them. Use our SDKs, the REST API, or gRPC API to build workers in any language.

Related guides:

Jobs & Queues Retries & DLQ SDKs

How Workers Work

Workers follow a simple claim-process-complete pattern. They poll for available jobs, process them, and report the result back to Spooled.

%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#ecfdf5', 'primaryTextColor': '#065f46', 'primaryBorderColor': '#10b981', 'lineColor': '#6b7280', 'signalColor': '#10b981'}}}%%
sequenceDiagram
  participant W as Worker
  participant API as Spooled API
  participant Q as Queue

  loop Every poll interval
    W->>API: POST /jobs/claim
    API->>Q: SELECT FOR UPDATE SKIP LOCKED
    Q-->>API: Job(s)
    API-->>W: job data
    
    W->>W: Process job
    
    alt Success
      W->>API: POST /jobs/complete
    else Failure
      W->>API: POST /jobs/fail
    end
  end

Worker Lifecycle

Register — Announce the worker with POST /api/v1/workers/register
Claim — Request one or more jobs from a queue
Process — Execute business logic with the job payload
Complete or Fail — Report the result back to Spooled
Repeat — Continue polling for more jobs

worker_id is optional — 1 to 128 characters from A-Z a-z 0-9 . _ - — but supplying the same one on every start turns registration into an upsert, so a restarting worker reuses its existing row. Hostname, pod name or container id all work.

Omit it and the server mints a new UUID each time. The abandoned row keeps occupying a slot in your plan’s worker cap until the stale-worker reaper clears it, about two minutes later — long enough that a crash-looping worker on a small plan can lock itself out of registering with 429 QUOTA_EXCEEDED. Re-registering an id you already own costs nothing against the cap; an id owned by a different organization returns 409.

Worker counters

GET /api/v1/workers and GET /api/v1/workers/{id} return two running totals for each worker next to its status and heartbeat: jobs_processed (jobs it completed) and jobs_failed (failures it reported, whether the job was scheduled for a retry or dead-lettered). The counters start at 0 for a worker row and count from API 0.1.122 on. They are per worker row, so a worker that restarts without a stable worker_id starts again at 0, and jobs released by a lease expiry or a stale-worker sweep are not counted as failures.

{
  "id": "emails-worker-1",
  "queue_name": "emails",
  "hostname": "worker-1",
  "status": "healthy",
  "current_jobs": 2,
  "max_concurrency": 5,
  "last_heartbeat": "2026-10-12T09:15:30Z",
  "jobs_processed": 1840,
  "jobs_failed": 12
}

Using SDKs

SDKs provide a high-level worker framework that handles polling, concurrency, and error handling automatically.

Worker with SDK

import { SpooledClient, SpooledWorker } from '@spooled/sdk';

const client = new SpooledClient({
  apiKey: process.env.SPOOLED_API_KEY!,
});

const worker = new SpooledWorker(client, {
  queueName: 'my-queue',
  concurrency: 10,
});

worker.process(async (ctx) => {
  await processJob(ctx.payload);
  return { success: true };
});

await worker.start();

import os
from spooled import SpooledClient
from spooled.worker import SpooledWorker

client = SpooledClient(api_key=os.environ["SPOOLED_API_KEY"])
worker = SpooledWorker(client, queue_name="my-queue", concurrency=10)

@worker.process
def handle_job(ctx):
    process_job(ctx.payload)
    return {"success": True}

worker.start()

<?php

use Spooled\SpooledClient;
use Spooled\Config\ClientOptions;
use Spooled\Worker\SpooledWorker;
use Spooled\Worker\WorkerConfig;
use Spooled\Worker\JobContext;

$client = new SpooledClient(new ClientOptions(
    apiKey: getenv('SPOOLED_API_KEY'),
));

$worker = new SpooledWorker($client, new WorkerConfig(
    queueName: 'my-queue',
    concurrency: 10,
));

$worker->process(function (JobContext $ctx): array {
    processJob($ctx->payload);
    return ['success' => true];
});

$worker->start();

See the SDKs documentation for language-specific examples:

Node.js: SpooledWorker with worker.process(async (ctx) => ...) (optional lifecycle events)
Python: @worker.process decorator to register a job handler
Go: worker.New(...) + worker.Process(...) with context cancellation
PHP: SpooledWorker with a handler callback (closure)

Using the REST API

For custom implementations or languages without an SDK, use the REST API directly.

Claim Jobs

# Claim up to 5 jobs
curl -X POST https://api.spooled.cloud/api/v1/jobs/claim \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queue_name": "my-queue",
    "worker_id": "worker-1",
    "limit": 5
  }'

// Claim jobs for processing
const { jobs } = await client.jobs.claim({
  queueName: 'my-queue',
  workerId: 'worker-1',
  limit: 10,
  leaseDurationSecs: 30,
});

for (const job of jobs) {
  console.log(`Claimed job: ${job.id}`);
  // Process the job...
}

# Claim jobs for processing
result = client.jobs.claim({
    "queue_name": "my-queue",
    "worker_id": "worker-1",
    "limit": 10,
    "lease_duration_secs": 30
})

for job in result.jobs:
    print(f"Claimed job: {job.id}")
    # Process the job...

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

claimed, err := client.Jobs().Claim(context.Background(), &resources.ClaimJobsRequest{
	QueueName: "my-queue",
	WorkerID:  "worker-1",
	Limit:     types.Int(10),
})
if err != nil {
	panic(err)
}

for _, job := range claimed.Jobs {
	fmt.Printf("Claimed job: %s\n", job.ID)
}

<?php

// Claim jobs for processing
$result = $client->jobs->claim([
    'queue' => 'my-queue',
    'workerId' => 'worker-1',
    'limit' => 10,
    'leaseDurationSecs' => 30,
]);

foreach ($result->jobs as $job) {
    echo "Claimed job: {$job->id}\n";
    // Process the job...
}

Complete Job

# Complete a job (echo lease_id from the claim response)
curl -X POST https://api.spooled.cloud/api/v1/jobs/job_xyz123/complete \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "worker_id": "worker-1",
    "lease_id": "lease_abc",
    "result": {"processed": true}
  }'

// Complete the job after processing (echo leaseId from claim)
await client.jobs.complete(job.id, {
  workerId: 'worker-1',
  leaseId: job.leaseId ?? undefined,
  result: { processed: true, outputUrl: 'https://...' },
});

# Complete the job after processing (echo lease_id from claim)
client.jobs.complete(job.id, {
    "worker_id": "worker-1",
    "lease_id": job.lease_id,
    "result": {"processed": True, "output_url": "https://..."}
})

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

err := client.Jobs().Complete(context.Background(), job.ID, &resources.CompleteJobRequest{
	WorkerID: "worker-1",
	LeaseID:  job.LeaseID,
	Result:   map[string]interface{}{"processed": true, "output_url": "https://..."},
})
if err != nil {
	panic(err)
}

fmt.Println("Job completed successfully")

<?php

// Complete the job after processing (echo leaseId from claim)
$client->jobs->complete($job->id, [
    'workerId' => 'worker-1',
    'leaseId' => $job->leaseId,
    'result' => ['processed' => true, 'outputUrl' => 'https://...'],
]);

Fail Job

# Fail a job (will retry if retries remaining; echo lease_id from claim)
curl -X POST https://api.spooled.cloud/api/v1/jobs/job_xyz123/fail \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "worker_id": "worker-1",
    "lease_id": "lease_abc",
    "error": "Connection timeout"
  }'

// Fail the job (will retry with exponential backoff)
await client.jobs.fail(job.id, {
  workerId: 'worker-1',
  leaseId: job.leaseId ?? undefined,
  error: 'Connection timeout to downstream service',
});

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

err := client.Jobs().Fail(context.Background(), job.ID, &resources.FailJobRequest{
	WorkerID: "worker-1",
	LeaseID:  job.LeaseID,
	Error:    "Connection timeout to downstream service",
})
if err != nil {
	panic(err)
}

fmt.Println("Job failed and will retry")

<?php

// Fail the job (will retry with exponential backoff)
$client->jobs->fail($job->id, [
    'workerId' => 'worker-1',
    'leaseId' => $job->leaseId,
    'error' => 'Connection timeout to downstream service',
]);

Extend Visibility (Heartbeat)

For long-running jobs, extend the visibility timeout to prevent re-delivery:

# Extend job lease (heartbeat; echo lease_id from claim)
curl -X POST https://api.spooled.cloud/api/v1/jobs/job_xyz123/heartbeat \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "worker_id": "worker-1",
    "lease_id": "lease_abc",
    "lease_duration_secs": 300
  }'

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

err := client.Jobs().Heartbeat(context.Background(), job.ID, &resources.HeartbeatRequest{
	WorkerID:         "worker-1",
	LeaseID:          job.LeaseID,
	LeaseDurationSec: types.Int(300), // 5 minutes
})
if err != nil {
	panic(err)
}

fmt.Println("Lease extended")

Using the gRPC API

For high-throughput workers, use the gRPC API with streaming. gRPC provides lower latency and enables real-time job streaming without polling. A StreamJobs stream keeps one job out with the worker at a time: the next job is delivered only after the worker has completed, failed or released the previous one (or its lease ran out), so open one stream per unit of concurrency.

Feature	REST API	gRPC API
Protocol	HTTP/1.1 + JSON	HTTP/2 + Protobuf
Job fetching	Polling (`/jobs/claim`)	Streaming (`StreamJobs`) or bidirectional (`ProcessJobs`)
Latency	~10-50ms per request	~1-5ms, persistent connection
Best for	Web apps, simple integrations	High-throughput workers

See the gRPC API documentation for examples using grpcurl or SDK helpers. Node, Go, and Python expose StreamJobs / ProcessJobs wrappers; PHP’s optional gRPC client is unary-focused today.

Lease fencing

Always send the claim/dequeue lease_id / leaseId on complete, fail, and heartbeat. It is required: REST answers 422 and gRPC INVALID_ARGUMENT without it. Built-in SpooledWorker runtimes do this for you; raw REST/gRPC loops must pass it explicitly.

gRPC jobs carry object payloads only

Over gRPC, payload and result are Protobuf Struct values: a job whose payload is an array, string, number, boolean or null reaches the worker with the payload missing, and integers of 2⁵³ or more arrive as rounded floats. REST keeps both exactly. See gRPC payloads for the workarounds.

On a gRPC Fail, retry: false moves the job to the dead-letter queue now, while leaving retry out retries it according to max_retries, as on REST. Register takes the same optional stable workerId as REST.

Visibility Timeout

When claimed, a job becomes invisible to other workers for the visibility timeout period. If the worker doesn't complete/fail the job in time, it becomes visible again for another worker to pick up.

Setting	Value	Description
Default lease (REST)	`30 seconds`	Good for most short-running jobs. gRPC claims that omit a lease get 300 seconds (server setting `GRPC_DEFAULT_LEASE_DURATION_SECS`).
Maximum lease	`1 hour (3600 seconds)`	Accepted range is 5-3600 seconds on claim and heartbeat. For longer jobs, send heartbeats to extend the lease.
Heartbeat interval	`visibility / 2`	Recommended heartbeat frequency

Dashboard Tip

Dashboard → Workers

What to look for:

→ Active workers and their queues
→ Last heartbeat timestamp
→ Jobs currently processing
→ Worker uptime

Actions:

Monitor worker health
Identify stale workers

Scaling Workers

Scale horizontally by running multiple worker instances. Workers coordinate automatically through Spooled — no external coordination (like Redis locks) needed.

%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#eff6ff', 'primaryTextColor': '#1e40af', 'primaryBorderColor': '#3b82f6', 'lineColor': '#6b7280'}}}%%
flowchart LR
  subgraph workers["Worker Pool"]
    W1["Worker 1"]
    W2["Worker 2"]
    W3["Worker 3"]
    WN["Worker N"]
  end
  
  Q[(Queue)] --> W1
  Q --> W2
  Q --> W3
  Q --> WN
  
  W1 --> R1[Result]
  W2 --> R2[Result]
  W3 --> R3[Result]
  WN --> RN[Result]

Concurrency

Control how many jobs a single worker processes in parallel:

I/O-bound jobs (API calls, email sending): Higher concurrency (10-100)
CPU-bound jobs (image processing): Match to available CPU cores
Memory-intensive jobs: Limit based on available memory

Best Practices

Idempotency

Jobs may be delivered more than once in edge cases (worker crash after processing but before completing). Design handlers to be idempotent by checking if the work was already done.

Graceful Shutdown

Handle shutdown signals (SIGTERM, SIGINT) to finish processing current jobs before exiting. SDKs provide built-in graceful shutdown support.

Structured Logging

Use structured logging for observability. Include job IDs and queue names in all log entries to make debugging easier.

Health Checks

Expose a health endpoint for orchestrators like Kubernetes. Workers should report unhealthy if they can't connect to Spooled.

Debug Worker Issues