Skip to content
On this page

Quickstart

Get up and running with Spooled Cloud in 5 minutes. This guide covers the basics of queuing jobs, processing them with workers, and handling failures gracefully.

See it in action

The live demos on the home page show real-time job processing, dependency workflows, retries into the dead-letter queue, and streaming — all running in your browser.

SDK Status

Node.js, Python, Go, and PHP SDKs are all production-ready! See SDKs for full documentation and examples.

API Options

Spooled Cloud provides two APIs:

  • REST API (port 8080) — HTTP/1.1 + JSON, best for web apps and simple integrations
  • gRPC API — HTTP/2 + Protobuf, best for high-throughput workers with streaming
gRPC endpoints
  • Spooled Cloud (TLS): grpc.spooled.cloud:443
  • Self-hosted / local: localhost:50051 (or GRPC_PORT)

This quickstart covers the REST API. See the gRPC documentation for high-performance worker implementations using gRPC streaming.

Prerequisites

Before you begin, you'll need:

  • A Spooled Cloud account — Sign up for free
  • An API key — Available in your dashboard
  • cURL, Node.js, Python, Go, or PHP — Any HTTP client will work

Step 1: Queue Your First Job

Jobs are the fundamental unit in Spooled. A job represents a task to be processed asynchronously, like sending an email, processing an image, or delivering a webhook.

curl -X POST https://api.spooled.cloud/api/v1/jobs \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queue_name": "my-queue",
    "payload": {
      "event": "user.created",
      "user_id": "usr_123",
      "email": "alice@example.com"
    },
    "idempotency_key": "user-created-usr_123"
  }'

The response includes the job ID and status. The idempotency_key prevents duplicate processing if you retry the request.

Step 2: Process Jobs with a Worker

Workers claim jobs from queues and process them. If processing fails, Spooled automatically retries with exponential backoff.

Worker Pattern: Claim → Process → Complete/Fail
# Worker loop: claim → process → complete/fail
# 1. Claim jobs
curl -X POST https://api.spooled.cloud/api/v1/jobs/claim \
  -H "Authorization: Bearer sp_live_..." \
  -H "Content-Type: application/json" \
  -d '{"queue_name": "my-queue", "worker_id": "worker-1", "limit": 5}'

# 2. Complete a job (echo lease_id from the claim response)
curl -X POST https://api.spooled.cloud/api/v1/jobs/job_xyz/complete \
  -H "Authorization: Bearer sp_live_..." \
  -H "Content-Type: application/json" \
  -d '{"worker_id": "worker-1", "lease_id": "lease_from_claim"}'

# 3. Or fail it (will retry)
curl -X POST https://api.spooled.cloud/api/v1/jobs/job_xyz/fail \
  -H "Authorization: Bearer sp_live_..." \
  -H "Content-Type: application/json" \
  -d '{"worker_id": "worker-1", "lease_id": "lease_from_claim", "error": "Processing failed"}'

Common Operations

Scheduled Jobs (Run Later)

Schedule a job to run at a specific time:

# Schedule a job for later
curl -X POST https://api.spooled.cloud/api/v1/jobs \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queue_name": "reminders",
    "payload": {"type": "cart-abandoned", "user_id": "usr_123"},
    "scheduled_at": "2030-01-15T09:00:00Z"
  }'

Cron Schedules (Recurring)

Create recurring jobs with cron expressions. Spooled accepts standard 5-field cron (min hour dom mon dow; seconds default to 0) and 6-field with a leading seconds field for second precision.

# Create a cron schedule
curl -X POST https://api.spooled.cloud/api/v1/schedules \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Daily Report",
    "cron_expression": "0 0 9 * * *",
    "timezone": "America/New_York",
    "queue_name": "reports",
    "payload_template": {"type": "daily_report"}
  }'

Schedule history

GET /api/v1/schedules/{id}/history lists the runs of a schedule, newest first (limit, default 50, at most 500). Each run is one job the schedule enqueued, and the run's status follows that job: a run does not read completed just because the job was created.

status The enqueued job is completed_at
pending Waiting (pending or scheduled) null
running Being processed by a worker (processing) null
completed Finished successfully When the job finished
failed Failed, dead-lettered or cancelled When the job ended

job_status is the job's own, raw status right now (pending, scheduled, processing, completed, failed, deadletter or cancelled), so a dead-lettered job shows status: "failed" with job_status: "deadletter". A run that never enqueued a job, or whose job your retention window has removed, has job_status: null and keeps the status it was stored with.

Bulk Enqueue

Enqueue multiple jobs in a single request (up to 100 jobs):

# Bulk enqueue up to 100 jobs
curl -X POST https://api.spooled.cloud/api/v1/jobs/bulk \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "queue_name": "notifications",
    "jobs": [
      {"payload": {"type": "email", "to": "a@example.com"}},
      {"payload": {"type": "sms", "to": "+1234567890"}},
      {"payload": {"type": "push", "token": "abc123"}}
    ]
  }'

Queue Management

Operation Endpoint
Pause queuePOST /api/v1/queues/:name/pause
Resume queuePOST /api/v1/queues/:name/resume
Get queue statsGET /api/v1/queues/:name/stats
Retry failed jobPOST /api/v1/jobs/:id/retry
Cancel jobDELETE /api/v1/jobs/:id

Real-time Updates

Get instant notifications when jobs complete or fail using WebSocket or Server-Sent Events (SSE).

WebSocket Connection
# WebSocket connections require a WebSocket client.
# Example using websocat (install: https://github.com/vi/websocat)
#
# 1) Exchange API key for a JWT access token (requires jq)
TOKEN=$(curl -s -X POST https://api.spooled.cloud/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"api_key":"sp_live_YOUR_API_KEY"}' | jq -r .access_token)
  
# 2) Connect (token goes in the query string)
websocat "wss://api.spooled.cloud/api/v1/ws?token=$TOKEN"

# 3) Subscribe by sending JSON commands over the socket.
# (Tip: see the Node.js/Python SDK tabs for a ready-to-run subscription example.)
Server-Sent Events
# Stream events via Server-Sent Events
curl -N \
  -H "Accept: text/event-stream" \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  "https://api.spooled.cloud/api/v1/events"

# Only job events for one queue
curl -N \
  -H "Accept: text/event-stream" \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  "https://api.spooled.cloud/api/v1/events?queue=my-queue&events=job.created,job.completed,job.failed"

# Queue counters (queue.stats every 5 s)
curl -N \
  -H "Accept: text/event-stream" \
  -H "Authorization: Bearer sp_live_YOUR_API_KEY" \
  "https://api.spooled.cloud/api/v1/events/queues/my-queue"

Event Types

Event Description
job.createdNew job enqueued
job.statusJob status changed (the Node SDK's WebSocket handlers call this one job.status_changed)
job.completedJob finished successfully
job.failedJob failed
queue.statsQueue counts, refreshed periodically
worker.heartbeatWorker heartbeat
worker.registeredWorker registered
worker.deregisteredWorker deregistered
system.healthSystem health, about every 10 seconds on the main event stream

There are no job.started, job.progress, queue.paused or queue.resumed events. See Real-time for payloads and what each stream emits.

Key Concepts

Idempotency

Use idempotency_key to prevent duplicate processing. If a job with the same key exists, the request returns the existing job.

Automatic Retries

Failed jobs retry automatically with exponential backoff (1s, 2s, 4s, 8s… capped at 60s). Set max_retries per job, or as a queue default with PUT /api/v1/queues/{name}/config.

Dead-Letter Queue

Jobs that exhaust retries move to the DLQ. Inspect failures and replay them with a single API call or bulk retry all DLQ jobs.

Priority Queues

Set priority from -100 to 100. Higher priority jobs are processed first. Default is 0.

Job Leases

A REST claim holds the job for 30 seconds by default (pass lease_duration_secs, 5–3600, to change it; gRPC Dequeue defaults to 5 minutes). If not completed before expiry, the job is released for another worker — heartbeat to extend the lease for long-running jobs. The SDK workers do this for you.

Real-time Updates

Stream job updates via WebSocket or SSE. Get instant notifications when jobs complete or fail without polling.

Troubleshooting

Invalid API Key

Account → API Keys

What to look for:

  • → Key starts with sp_live_ or sp_test_ (or legacy sk_)
  • → Key hasn't been revoked
  • → Using correct organization's key

Actions:

  • Generate a new key if needed
  • Check environment variables are set correctly

Rate Limited

Account → Usage

What to look for:

  • → Current request rate vs limit
  • → Retry-After header value

Actions:

  • Implement exponential backoff
  • Use bulk operations where possible
  • Consider upgrading your plan

Job Stuck in Processing

Dashboard → Jobs → Processing

What to look for:

  • → Worker heartbeat status
  • → Lease expiration time
  • → Worker logs for errors

Actions:

  • Jobs with expired leases auto-recover
  • Force-fail stuck jobs via API
  • Check worker health

Next Steps

Last updated 2026-10-16