Jobs & Queues

Jobs are the fundamental building blocks of Spooled. Learn how to queue, process, and manage asynchronous work reliably at any scale.

Job Lifecycle

Every job follows a predictable lifecycle from creation to completion. Understanding this lifecycle is key to building reliable systems.

%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#ecfdf5', 'primaryTextColor': '#065f46', 'primaryBorderColor': '#10b981', 'lineColor': '#6b7280'}}}%%
stateDiagram-v2
    [*] --> pending: Job created
    [*] --> scheduled: Created with a future scheduled_at
    scheduled --> pending: Time reached
    scheduled --> cancelled: Cancelled
    pending --> processing: Worker claims
    processing --> completed: Success
    processing --> scheduled: Error, retries left
    processing --> deadletter: Error, retries exhausted
    pending --> cancelled: Cancelled
    deadletter --> pending: Retry from DLQ
    completed --> [*]

Job States

State	Description	Transitions
`pending`	Job is waiting to be processed	→ processing (by worker)
`scheduled`	Waiting until `scheduled_at`. A future job starts here, and an automatic retry waits here until its run time, then becomes `pending`	→ pending (when `scheduled_at` is reached), cancelled
`processing`	Worker is processing the job	→ completed, scheduled (automatic retry until its run time, then pending), deadletter
`completed`	Job finished successfully	Terminal state
`deadletter`	Job in dead-letter queue	→ pending (retry from DLQ)
`cancelled`	Cancelled with `DELETE /api/v1/jobs/{id}` before it ran	Terminal state

Dashboard Tip

Dashboard → Jobs

What to look for:

→ Filter jobs by status (pending, scheduled, processing, completed, deadletter, cancelled)
→ View job payload and metadata
→ See retry count and last error

Creating Jobs

Use the SDK or REST API to create jobs. Each job belongs to a queue and carries a payload that your workers will process.

curl -X POST https://api.spooled.cloud/api/v1/jobs \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queue_name": "my-queue",
    "payload": {
      "event": "user.created",
      "user_id": "usr_123",
      "email": "alice@example.com"
    },
    "idempotency_key": "user-created-usr_123"
  }'

import { SpooledClient } from '@spooled/sdk';

const client = new SpooledClient({
  apiKey: process.env.SPOOLED_API_KEY!,
});

const userId = 'usr_123';

// Create a job
const { id } = await client.jobs.create({
  queueName: 'email-notifications',
  payload: {
    to: 'user@example.com',
    subject: 'Welcome!',
    template: 'welcome',
  },
  idempotencyKey: `welcome-${userId}`,
  maxRetries: 5,
});

console.log(`Created job: ${id}`);

from spooled import SpooledClient
import os

client = SpooledClient(api_key=os.environ["SPOOLED_API_KEY"])

image_id = "img_123"

# Create a background job
result = client.jobs.create({
    "queue_name": "image-processing",
    "payload": {
        "image_url": "https://example.com/image.jpg",
        "operations": ["resize", "compress"],
        "output_format": "webp"
    },
    "idempotency_key": f"process-image-{image_id}",
    "max_retries": 3
})

print(f"Created job: {result.id}")

client.close()

package main

import (
	"context"
	"fmt"
	"os"

	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

func ptr[T any](v T) *T { return &v }

func main() {
	client, err := spooled.NewClient(spooled.WithAPIKey(os.Getenv("SPOOLED_API_KEY")))
	if err != nil {
		panic(err)
	}

	resp, err := client.Jobs().Create(context.Background(), &resources.CreateJobRequest{
		QueueName:      "my-queue",
		Payload:        map[string]any{"key": "value"},
		IdempotencyKey: ptr("unique-key"),
		MaxRetries:     ptr(3),
	})
	if err != nil {
		panic(err)
	}

	fmt.Printf("Created job: %s\n", resp.ID)
}

<?php

use Spooled\SpooledClient;
use Spooled\Config\ClientOptions;

$client = new SpooledClient(new ClientOptions(
    apiKey: getenv('SPOOLED_API_KEY'),
));

$userId = 'usr_123';

// Create a job
$job = $client->jobs->create([
    'queue' => 'email-notifications',
    'payload' => [
        'to' => 'user@example.com',
        'subject' => 'Welcome!',
        'template' => 'welcome',
    ],
    'idempotencyKey' => "welcome-{$userId}",
    'maxRetries' => 5,
]);

echo "Created job: {$job->id}\n";

Job Properties

Property	Type	Description
`queue_name`	string	Queue name (alphanumeric + hyphens). Required on create.
`payload`	object	JSON payload (plan caps: 64 KB free → 1 MB Pro/Enterprise; platform hard cap 1 MiB)
`idempotency_key`	string	Prevents duplicate jobs (optional but recommended)
`max_retries`	number	Max retry attempts. Omitted → server settings default (typically 3). Explicit `0` allowed on REST (first fail can DLQ).
`timeout_seconds`	number	Stored with the job, but not enforced: neither the server nor the SDK workers stop a handler that runs past it. Enforce your own time limit in the handler; the claim lease (`lease_duration_secs`, extended by heartbeats) is what controls re-delivery.
`scheduled_at`	Date	When to process (null = immediately)
`priority`	number	Higher values processed first (default: 0)
`tags`	array \| object	Optional tags for filtering (stored as JSON)
`completion_webhook`	string (URL)	Optional per-job callback URL.
`completion_webhook_secret`	string	Optional HMAC secret (backend 0.1.109+). When set, delivery includes `X-Spooled-Timestamp` + `X-Spooled-Signature` like org outgoing webhooks. Omit for unsigned callbacks.

List and stats filters accept canonical queue_name or the alias queue (e.g. GET /api/v1/jobs?queue=emails and GET /api/v1/jobs/stats?queue_name=emails).

Filtering jobs (including tags)

GET /api/v1/jobs takes these query parameters. They combine with AND, and an empty parameter (?status=) is no filter:

Parameter	Meaning
`queue_name` (alias `queue`)	One queue. Letters, digits, `-`, `_` and `.` only
`status`	One status or a comma-separated list: `pending`, `scheduled`, `processing`, `completed`, `failed`, `deadletter`, `cancelled` (case-insensitive), e.g. `status=failed,deadletter`
`q`	Search text (case-insensitive, at most 255 bytes): matches a job whose id starts with it, or whose queue name or job type contains it
`job_type`	Exact `payload.job_type`
`created_after`, `created_before`	RFC 3339 instants: created at or after / before. Percent-encode a `+` offset as `%2B`
`tag`	A single tag (any characters, up to 1024 bytes)
`order_by`, `order_dir`	`created_at` (default), `updated_at`, `scheduled_at`, `completed_at`, `priority`, `status`, `queue_name`, `retry_count`; `desc` (default) or `asc`
`limit`, `offset`	Page size (default 50, at most 100) and rows to skip

# Failed and dead-lettered jobs in one queue since a given time
curl -G https://api.spooled.cloud/api/v1/jobs \
  -H "Authorization: Bearer YOUR_API_KEY" \
  --data-urlencode "queue_name=emails" \
  --data-urlencode "status=failed,deadletter" \
  --data-urlencode "created_after=2026-10-11T00:00:00Z"

# Jobs tagged "urgent", and DLQ jobs tagged "urgent"
curl "https://api.spooled.cloud/api/v1/jobs?tag=urgent" \
  -H "Authorization: Bearer YOUR_API_KEY"
curl "https://api.spooled.cloud/api/v1/jobs/dlq?tag=urgent" \
  -H "Authorization: Bearer YOUR_API_KEY"

An invalid status (any entry of the list), queue_name, tag, q, job_type, created_after, created_before, order_by or order_dir is a 400 VALIDATION_ERROR. It is not ignored: before API 0.1.121 an invalid filter quietly returned the unfiltered list. Rows that tie on the sort column are ordered by job id, so offset pages never repeat or skip a job. The body is a plain array; its X-Has-More header says whether another page follows (see Pagination). GET /api/v1/jobs/dlq takes the same parameters except status.

Slow lists stop with 400 QUERY_TIMEOUT

A job list runs under a statement timeout (5 seconds by default). A list the database cannot answer in time is stopped and answered 400 with code QUERY_TIMEOUT. Retrying the same request would time out again: narrow it with queue_name, status, job_type, a created window or a longer q. The job-type part of q looks at your first 1,000 distinct job types only; job_type (exact), the id prefix and the queue name still find the rest.

Job history

GET /api/v1/jobs/{id}/history returns the lifecycle events recorded for one job, oldest first (order_dir=desc for newest first; order works as an alias). limit is 1 to 500 (default 100), offset skips events, and the X-Has-More header says whether more follow.

curl "https://api.spooled.cloud/api/v1/jobs/job_550e8400e29b41d4/history?order_dir=desc&limit=20" \
  -H "Authorization: Bearer YOUR_API_KEY"

[
  {
    "id": "b2a4e7d0-5c1f-4a38-9e63-0d7f4c2a9b11",
    "job_id": "job_550e8400e29b41d4",
    "event_type": "retry_scheduled",
    "worker_id": null,
    "details": {
      "error": "SMTP timeout",
      "retry_count": 1,
      "next_run": "2026-10-12T09:00:09Z",
      "organization_id": "org_xxxxx"
    },
    "created_at": "2026-10-12T09:00:07Z"
  },
  {
    "id": "6f0c1a52-9a0e-4c0e-8d52-3c0f6a7f1d10",
    "job_id": "job_550e8400e29b41d4",
    "event_type": "processing",
    "worker_id": "emails-worker-1",
    "details": { "worker_id": "emails-worker-1", "lease_id": "lease_7f3c9a", "organization_id": "org_xxxxx" },
    "created_at": "2026-10-12T09:00:01Z"
  }
]

event_type is processing, completed, retry_scheduled or deadlettered. Events are recorded by REST claim, complete and fail and by the scheduler's dead-letter sweeps, so the history is sparse for jobs handled over gRPC and a job has no created event. A job of another organization, an unknown id and a job in a queue outside the key's scope all answer 404.

Claiming & Processing Jobs

Workers claim jobs to process them. Claimed jobs are locked for a visibility timeout to prevent duplicate processing.

# Claim up to 5 jobs
curl -X POST https://api.spooled.cloud/api/v1/jobs/claim \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queue_name": "my-queue",
    "worker_id": "worker-1",
    "limit": 5
  }'

// Claim jobs for processing
const { jobs } = await client.jobs.claim({
  queueName: 'my-queue',
  workerId: 'worker-1',
  limit: 10,
  leaseDurationSecs: 30,
});

for (const job of jobs) {
  console.log(`Claimed job: ${job.id}`);
  // Process the job...
}

# Claim jobs for processing
result = client.jobs.claim({
    "queue_name": "my-queue",
    "worker_id": "worker-1",
    "limit": 10,
    "lease_duration_secs": 30
})

for job in result.jobs:
    print(f"Claimed job: {job.id}")
    # Process the job...

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

claimed, err := client.Jobs().Claim(context.Background(), &resources.ClaimJobsRequest{
	QueueName: "my-queue",
	WorkerID:  "worker-1",
	Limit:     types.Int(10),
})
if err != nil {
	panic(err)
}

for _, job := range claimed.Jobs {
	fmt.Printf("Claimed job: %s\n", job.ID)
}

<?php

// Claim jobs for processing
$result = $client->jobs->claim([
    'queue' => 'my-queue',
    'workerId' => 'worker-1',
    'limit' => 10,
    'leaseDurationSecs' => 30,
]);

foreach ($result->jobs as $job) {
    echo "Claimed job: {$job->id}\n";
    // Process the job...
}

Important: Visibility Timeout

If a worker doesn't complete or fail a job within the visibility timeout, the job becomes available for other workers. Set this higher than your expected processing time.

Lease fencing

Claim/dequeue responses include a lease_id (SDK: leaseId). Pass it back on complete, fail, and heartbeat: it is required (a request without it is a 422 VALIDATION_ERROR, and INVALID_ARGUMENT over gRPC), and it keeps a stale worker after lease expiry from acking a job another worker already holds (409 LEASE_EXPIRED). Official worker runtimes do this automatically.

Completing Jobs

# Complete a job (echo lease_id from the claim response)
curl -X POST https://api.spooled.cloud/api/v1/jobs/job_xyz123/complete \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "worker_id": "worker-1",
    "lease_id": "lease_abc",
    "result": {"processed": true}
  }'

// Complete the job after processing (echo leaseId from claim)
await client.jobs.complete(job.id, {
  workerId: 'worker-1',
  leaseId: job.leaseId ?? undefined,
  result: { processed: true, outputUrl: 'https://...' },
});

# Complete the job after processing (echo lease_id from claim)
client.jobs.complete(job.id, {
    "worker_id": "worker-1",
    "lease_id": job.lease_id,
    "result": {"processed": True, "output_url": "https://..."}
})

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

err := client.Jobs().Complete(context.Background(), job.ID, &resources.CompleteJobRequest{
	WorkerID: "worker-1",
	LeaseID:  job.LeaseID,
	Result:   map[string]interface{}{"processed": true, "output_url": "https://..."},
})
if err != nil {
	panic(err)
}

fmt.Println("Job completed successfully")

<?php

// Complete the job after processing (echo leaseId from claim)
$client->jobs->complete($job->id, [
    'workerId' => 'worker-1',
    'leaseId' => $job->leaseId,
    'result' => ['processed' => true, 'outputUrl' => 'https://...'],
]);

Failing Jobs

# Fail a job (will retry if retries remaining; echo lease_id from claim)
curl -X POST https://api.spooled.cloud/api/v1/jobs/job_xyz123/fail \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "worker_id": "worker-1",
    "lease_id": "lease_abc",
    "error": "Connection timeout"
  }'

// Fail the job (will retry with exponential backoff)
await client.jobs.fail(job.id, {
  workerId: 'worker-1',
  leaseId: job.leaseId ?? undefined,
  error: 'Connection timeout to downstream service',
});

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

err := client.Jobs().Fail(context.Background(), job.ID, &resources.FailJobRequest{
	WorkerID: "worker-1",
	LeaseID:  job.LeaseID,
	Error:    "Connection timeout to downstream service",
})
if err != nil {
	panic(err)
}

fmt.Println("Job failed and will retry")

<?php

// Fail the job (will retry with exponential backoff)
$client->jobs->fail($job->id, [
    'workerId' => 'worker-1',
    'leaseId' => $job->leaseId,
    'error' => 'Connection timeout to downstream service',
]);

Heartbeat (Extend Lease)

For long-running jobs, extend the lease to prevent the job from being released:

# Extend job lease (heartbeat; echo lease_id from claim)
curl -X POST https://api.spooled.cloud/api/v1/jobs/job_xyz123/heartbeat \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "worker_id": "worker-1",
    "lease_id": "lease_abc",
    "lease_duration_secs": 300
  }'

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

err := client.Jobs().Heartbeat(context.Background(), job.ID, &resources.HeartbeatRequest{
	WorkerID:         "worker-1",
	LeaseID:          job.LeaseID,
	LeaseDurationSec: types.Int(300), // 5 minutes
})
if err != nil {
	panic(err)
}

fmt.Println("Lease extended")

Batch Operations

Queue multiple jobs in a single API call for better performance. All jobs in a batch share the same idempotency key scope.

# Bulk enqueue up to 100 jobs
curl -X POST https://api.spooled.cloud/api/v1/jobs/bulk \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queue_name": "notifications",
    "jobs": [
      {"payload": {"type": "email", "to": "a@example.com"}},
      {"payload": {"type": "sms", "to": "+1234567890"}},
      {"payload": {"type": "push", "token": "abc123"}}
    ]
  }'

// Bulk enqueue multiple jobs (up to 100)
const result = await client.jobs.bulkEnqueue({
  queueName: 'notifications',
  jobs: [
    { payload: { type: 'email', to: 'a@example.com' } },
    { payload: { type: 'sms', to: '+1234567890' } },
    { payload: { type: 'push', token: 'abc123' } },
  ],
  defaultMaxRetries: 5,
});

console.log(`Enqueued ${result.successCount} jobs`);

# Bulk enqueue multiple jobs (up to 100)
result = client.jobs.bulk_enqueue({
    "queue_name": "notifications",
    "jobs": [
        {"payload": {"type": "email", "to": "a@example.com"}},
        {"payload": {"type": "sms", "to": "+1234567890"}},
        {"payload": {"type": "push", "token": "abc123"}},
    ],
    "default_max_retries": 5
})

print(f"Enqueued {result.success_count} jobs")

import (
	"context"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

result, err := client.Jobs().BulkEnqueue(context.Background(), &resources.BulkEnqueueRequest{
	QueueName: "notifications",
	Jobs: []resources.BulkJobItem{
		{Payload: map[string]interface{}{"type": "email", "to": "a@example.com"}},
		{Payload: map[string]interface{}{"type": "sms", "to": "+1234567890"}},
		{Payload: map[string]interface{}{"type": "push", "token": "abc123"}},
	},
})
if err != nil {
	panic(err)
}

fmt.Printf("Enqueued %d jobs\n", result.Succeeded)

<?php

// Bulk enqueue multiple jobs (up to 100)
$result = $client->jobs->bulkEnqueue([
    'queueName' => 'notifications',
    'jobs' => [
        ['payload' => ['type' => 'email', 'to' => 'a@example.com']],
        ['payload' => ['type' => 'sms', 'to' => '+1234567890']],
        ['payload' => ['type' => 'push', 'token' => 'abc123']],
    ],
    'defaultMaxRetries' => 5,
]);

echo "Enqueued {$result->successCount} jobs\n";

Scheduled Jobs

Schedule jobs for future execution. Perfect for reminders, delayed notifications, or recurring tasks.

# Schedule a job for later
curl -X POST https://api.spooled.cloud/api/v1/jobs \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queue_name": "reminders",
    "payload": {"type": "cart-abandoned", "user_id": "usr_123"},
    "scheduled_at": "2030-01-15T09:00:00Z"
  }'

// Schedule a job to run in 24 hours
await client.jobs.create({
  queueName: 'reminders',
  payload: { userId: 'usr_123', type: 'cart-abandoned' },
  scheduledAt: new Date(Date.now() + 24 * 60 * 60 * 1000),
  idempotencyKey: `reminder-${userId}-cart`,
});

from datetime import datetime, timedelta, timezone

user_id = "usr_123"

# Schedule a job to run in 24 hours (timezone-aware; datetime.utcnow() is deprecated)
client.jobs.create({
    "queue_name": "reminders",
    "payload": {"user_id": user_id, "type": "cart-abandoned"},
    "scheduled_at": datetime.now(timezone.utc) + timedelta(hours=24),
    "idempotency_key": f"reminder-{user_id}-cart"
})

import (
	"github.com/spooled-cloud/spooled-sdk-go/spooled/types"
	"context"
	"time"
	"github.com/spooled-cloud/spooled-sdk-go/spooled"
	"github.com/spooled-cloud/spooled-sdk-go/spooled/resources"
)

client, _ := spooled.NewClient(spooled.WithAPIKey("sp_live_YOUR_API_KEY"))

scheduledTime := time.Now().Add(24 * time.Hour)
resp, err := client.Jobs().Create(context.Background(), &resources.CreateJobRequest{
	QueueName:      "reminders",
	Payload:        map[string]interface{}{"user_id": "usr_123", "type": "cart-abandoned"},
	ScheduledAt:    &scheduledTime,
	IdempotencyKey: types.String("reminder-usr_123-cart"),
})
if err != nil {
	panic(err)
}

fmt.Printf("Scheduled job: %s\n", resp.ID)

<?php

// Schedule a job to run in 24 hours
$scheduledAt = (new DateTime('+24 hours'))->format(DateTime::ATOM);

$client->jobs->create([
    'queue' => 'reminders',
    'payload' => ['userId' => 'usr_123', 'type' => 'cart-abandoned'],
    'scheduledAt' => $scheduledAt,
    'idempotencyKey' => "reminder-{$userId}-cart",
]);

Idempotency

Idempotency keys prevent duplicate job processing. When you create a job with an idempotency key:

If no job exists with that key, a new job is created
If a job exists and is still pending/processing, the existing job is returned
If a job exists and completed, the completed job is returned (no duplicate)

Best Practice

Always use meaningful idempotency keys derived from your business logic, like process-order-{orderId} or send-welcome-email-{userId}.

Working with Queues

Queues are logical groupings of related jobs. Use separate queues for different job types to enable independent scaling and monitoring.

Queue Naming

Use lowercase letters, numbers, and hyphens
Maximum 64 characters
Be descriptive: payment-processing, email-notifications

Recommended Queue Structure

High-priority queue — Critical operations (payments, security alerts)
Default queue — Standard background jobs
Low-priority queue — Analytics, cleanup, non-urgent tasks

Dashboard Tip

Dashboard → Queues

What to look for:

→ Jobs pending vs processing
→ Throughput rate
→ Pause/resume controls

Actions:

Pause a queue during maintenance
Monitor queue depth for scaling decisions

Next Steps

Configure retry behavior — Customize backoff and failure handling
Build production workers — Best practices for reliable processing
Real-time job updates — Stream job status via WebSocket