Skip to content
On this page

Troubleshooting

Find answers to common questions and solutions to frequently encountered issues. If you can't find what you're looking for, contact support.

Jump to:

Error Codes Jobs Not Processing Authentication Errors Webhook Issues Worker Problems Rate Limiting

Common Error Codes

Every error response is JSON with at least code and message, e.g. {"code":"NOT_FOUND","message":"Resource not found"}. Some add fields: details (on validation errors, an array of { field, message, code } entries), resource/limit on quota errors. Branch on code, not on the message text.

Code Body code Meaning Action
400 VALIDATION_ERROR / INVALID_JSON / BAD_REQUEST / QUERY_TIMEOUT Bad Request A rule was broken (field length, allowed characters, ranges, an invalid list filter) or the body is empty or not valid JSON. Read details[] for the field. QUERY_TIMEOUT: a job list ran into its statement timeout; retrying times out again, so narrow it (queue_name, status, job_type, a created window, a longer q)
401 UNAUTHORIZED / AUTHENTICATION_FAILED Unauthorized Send the key in the Authorization: Bearer header (not the query string) and verify it is valid
403 ACCESS_DENIED Forbidden The key's queue scope excludes this queue, or the route needs an unrestricted key
404 NOT_FOUND Not Found Verify resource ID and endpoint URL
409 CONFLICT / LEASE_EXPIRED Conflict / lease expired CONFLICT: resource already exists, or the job is already finished (e.g. a second complete). LEASE_EXPIRED: stop work — the lease timed out or your lease_id is no longer the job's current lease
413 PAYLOAD_TOO_LARGE Payload too large Shrink payload or upgrade plan; see limits
422 VALIDATION_ERROR Unprocessable Entity Valid JSON, but a required field is missing or has the wrong type (details[].code is missing_field or invalid_type), for example no lease_id on a job complete, fail or heartbeat. Rule violations on a present field are 400
429 RATE_LIMIT_EXCEEDED / QUOTA_EXCEEDED Rate limited / quota exceeded RATE_LIMIT_EXCEEDED (has Retry-After): wait that many seconds, then retry; for too many open realtime streams there is no Retry-After, close a stream first. QUOTA_EXCEEDED (a plan limit, no Retry-After): retrying will not help, so free capacity or upgrade to the plan in upgrade_to. See two kinds of 429
500 INTERNAL_ERROR / DATABASE_ERROR / CACHE_ERROR Server Error Retry with backoff, contact support if persistent
503 — Service Unavailable Check status page, retry later

Jobs Not Processing

Jobs stay in "pending" status

Symptom / confirm

Jobs are created successfully but never move to "processing" state.

  • No workers running: Ensure you have at least one worker connected to the queue
  • Workers on different queue: Verify workers are listening to the correct queue name
  • Workers paused: Check if the queue or workers are paused in the dashboard
  • Scheduled for future: Jobs with scheduled_at won't process until that time

Fix

How to diagnose:

Bash 
# Check if workers are registered
curl -H "Authorization: Bearer YOUR_API_KEY" \
  https://api.spooled.cloud/api/v1/workers

# Check job details
curl -H "Authorization: Bearer YOUR_API_KEY" \
  https://api.spooled.cloud/api/v1/jobs/JOB_ID

Jobs stuck in "processing" state

Symptom / confirm

Jobs move to processing but never complete.

  • Worker crashed: Worker died without completing the job
  • Long-running job: Job is taking longer than expected
  • Heartbeat timeout: Worker didn't send heartbeats (lease expired)

Fix

Jobs with expired leases are automatically returned to pending. If this happens frequently:

  • Increase heartbeat frequency in your worker
  • Implement proper heartbeating for long-running jobs
  • Check worker logs for crashes

Authentication Errors

401 Unauthorized

Symptom / confirm

API requests return 401 Unauthorized.

  • Missing API key: Ensure Authorization: Bearer YOUR_API_KEY header is present
  • Credential in the query string: REST rejects ?api_key= and ?token= — a previously working integration that passed either one now gets a bare 401
  • Invalid API key: Key may be revoked or incorrectly copied
  • Wrong environment: Using test key with production endpoint or vice versa

Fix

Move the credential out of the URL and into the Authorization: Bearer … header. Only /api/v1/ws, /api/v1/events, /api/v1/events/jobs/{id} and /api/v1/events/queues/{name} still take it in the query string — browser EventSource and WebSocket clients cannot set headers. Quick check:

Bash 
# Verify your API key works: any authenticated read will do.
# 200 = key accepted, 401 = key invalid or revoked.
# (/health is public and says nothing about your key.)
curl -i -H "Authorization: Bearer YOUR_API_KEY" \
  "https://api.spooled.cloud/api/v1/jobs?limit=1"

403 Forbidden

Symptom / confirm

API requests return 403 Forbidden.

  • Organization mismatch: Accessing resources from a different organization
  • Queue scope: the key's queues list does not include this queue, or the route needs an unrestricted key (API keys, organization settings)
  • Rate limited: Too many failed authentication attempts

Fix

  • Use an API key from the same organization as the resource
  • Use a key with the required scope
  • Stop hammering auth after failures (failed attempts can rate-limit)

JWT Token Expired

Symptom / confirm

Dashboard sessions expire frequently, workers can't authenticate.

Fix

  • For dashboard: Clear cookies and log in again
  • For workers: Ensure workers refresh tokens before expiration
  • Check server clock synchronization (NTP)

Webhook Issues

Webhooks not being received

Symptom / confirm

Jobs complete but webhook notifications never arrive.

  • Endpoint auto-disabled: after 20 consecutive failed deliveries the webhook is switched off — enabled is false and last_status is auto_disabled. Check this first: fixing the URL, cert or status code changes nothing while it is off
  • Verify webhook URL is correctly configured in organization settings
  • Ensure your endpoint is publicly accessible (not localhost)
  • Check that your endpoint returns 2xx status code
  • Verify SSL certificate is valid (we don't deliver to invalid certs in production)

Fix

  • Re-enable the endpoint once it is healthy: PUT /api/v1/outgoing-webhooks/{id} with {"enabled": true}. That counts against your plan webhook cap, so it can return 429 QUOTA_EXCEEDED — see outgoing webhook delivery
  • Update the webhook URL in organization settings
  • Expose the endpoint (tunnel / public host) and retest delivery
  • Return 2xx quickly; fix cert chain if TLS fails in production

Webhook signature validation failing

Symptom / confirm

Receiving webhooks but signature verification fails.

  • Wrong secret: Using incorrect webhook secret for verification
  • Secret was cleared: an update that sent "secret": null removes it, and deliveries then arrive with no X-Spooled-Signature header at all
  • Encoding issues: Not using raw request body for signature verification
  • Timestamp tolerance: Clock drift causing timestamp validation to fail

Fix

Correct verification pattern:

JavaScript 
// Node.js — verify an outgoing Spooled webhook.
// Spooled signs "<X-Spooled-Timestamp>.<raw body>" with HMAC-SHA256 and sends
// "sha256=<hex>" in X-Spooled-Signature. Use the RAW body bytes as received,
// not a re-serialized object (e.g. express.raw({ type: 'application/json' })).
const crypto = require('crypto');

function verifySpooledWebhook(rawBody, headers, secret, toleranceSecs = 300) {
  const timestamp = headers['x-spooled-timestamp'];
  const signature = headers['x-spooled-signature'];
  if (!timestamp || !signature) return false;

  // Each delivery attempt is signed with a fresh timestamp; reject stale ones.
  const age = Math.abs(Date.now() / 1000 - Number(timestamp));
  if (!(age <= toleranceSecs)) return false;

  const expected =
    'sha256=' +
    crypto.createHmac('sha256', secret).update(timestamp + '.' + rawBody).digest('hex');

  const a = Buffer.from(signature);
  const b = Buffer.from(expected);
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

Webhook delivery timeouts

Symptom / confirm

Webhooks marked as failed with timeout errors.

Fix

  • Ensure your endpoint responds within 10 seconds
  • Process webhooks asynchronously (respond 200 immediately, process later)
  • Check your server isn't overloaded

Worker Problems

Workers disconnecting frequently

Symptom / confirm

Workers show as unhealthy, reconnecting constantly.

  • Network instability: Unstable connection to Spooled Cloud
  • Resource exhaustion: Worker running out of memory/CPU
  • Firewall issues: gRPC connections being terminated

Fix

  • Implement reconnection logic with exponential backoff
  • Monitor worker resource usage
  • Ensure firewalls allow long-lived HTTP/2 connections

Worker can't register (429 QUOTA_EXCEEDED)

Symptom / confirm

A restarting worker is refused with 429 and "resource": "workers", while the dashboard shows workers you no longer run.

  • No stable worker_id: each start mints a fresh UUID and abandons the previous row, which keeps holding a slot in the plan worker cap until the stale-worker reaper clears it (about two minutes)
  • Crash loop: a worker restarting faster than that leaks a row per restart and locks itself out

Fix

  • Pass a stable worker_id on POST /api/v1/workers/register — hostname, pod name or container id. Registration becomes an upsert and re-registering your own id is not charged against the cap
  • Deregister on shutdown so the slot is released immediately
  • Wait out the reaper (about two minutes) to clear rows already leaked, then retry

Workers not receiving jobs

Symptom / confirm

Worker is connected but never receives jobs.

  • Verify worker is subscribed to the correct queue name(s)
  • Check there are pending jobs in that queue
  • Ensure worker concurrency is set correctly (not 0)
  • Verify worker health status in dashboard

Fix

  • Align worker queue subscription with the queue that has pending jobs
  • Raise concurrency above 0
  • Confirm health is green in the dashboard before retrying

Duplicate job processing

Symptom / confirm

Same job processed multiple times.

  • Worker crashed during processing
  • Network partition: Job lease expired
  • Missing acknowledgment

Fix

  • Implement idempotent job handlers
  • Use longer lease durations
  • Always complete or fail jobs explicitly

Rate Limiting

429 Too Many Requests

Symptom / confirm

API requests return 429 status code.

Two conditions return 429:

  • Per-second rate limiting — too many requests in a short window (includes a Retry-After header, the wait in seconds). Body code: RATE_LIMIT_EXCEEDED. Every authenticated response also carries X-RateLimit-Limit, -Remaining, -Reset (a Unix time in seconds, not a wait) and -Scope: see rate-limit headers.
  • Plan quota / limit exceeded — body carries "code": "QUOTA_EXCEEDED" with resource, current, limit, plan and upgrade_to. There is no Retry-After header and retrying the same request will not help. See Two kinds of 429 and Rate Limits & Quotas.

Plan rate limits (req/sec · burst): Free 5 · 10, Starter 25 · 50, Pro 100 · 200, Enterprise 500 · 1000. Full table: plan comparison.

Fix

  • On RATE_LIMIT_EXCEEDED, implement exponential backoff, or wait for Retry-After seconds
  • On QUOTA_EXCEEDED, do not retry in a loop: free capacity (finish or delete jobs, remove unused keys or schedules) or upgrade to upgrade_to
  • Watch X-RateLimit-Remaining and slow down before it reaches 0
  • Use bulk operations (e.g., bulkEnqueue) instead of individual requests
  • Contact support for higher limits if needed

Handling rate limits:

JavaScript 
// Exponential backoff example
async function apiCallWithRetry(fn, maxRetries = 5) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (error) {
      // QUOTA_EXCEEDED is a plan limit, not a rate limit: retrying cannot fix it
      const retryable = error.statusCode === 429 && error.code !== 'QUOTA_EXCEEDED';
      if (retryable && i < maxRetries - 1) {
        const delay = Math.pow(2, i) * 1000; // 1s, 2s, 4s, 8s, 16s
        await new Promise((r) => setTimeout(r, delay));
        continue;
      }
      throw error;
    }
  }
}

Debugging Tips

Enable verbose logging

For SDK debugging, enable verbose mode:

JavaScript 
// Node.js SDK
const client = new SpooledClient({
  apiKey: 'YOUR_API_KEY',
  debug: true,
});
Python 
# Python SDK
import logging

logging.basicConfig(level=logging.DEBUG)
client = SpooledClient(api_key='YOUR_API_KEY')

Use the Dashboard

The Spooled Dashboard provides:

  • Real-time job status and history
  • Worker health monitoring
  • Queue metrics and throughput graphs
  • DLQ management for failed jobs
  • Webhook delivery logs

Still Stuck?

Need Help?

If you can't find the answer here, we're happy to help:

When contacting support, please include:

  • Your organization ID (visible in Dashboard)
  • Relevant job IDs or timestamps
  • Error messages (full response if possible)
  • Steps to reproduce the issue
  • SDK version (if using an SDK)

Last updated 2026-10-16