{"protocol":"clank-doc/1","frameworkVersion":"0.9.3","slug":"jobs-and-cron","title":"Durable jobs and cron","description":"Clank runs slow, failure prone, delayed, and scheduled work outside the web request process. The queue is durable in the application's isolated SQLite database, enqueue can share the mutation transaction, and any number of worker processes ","group":{"id":"full-stack","title":"Full stack"},"url":"https://docs.clank.run/docs/jobs-and-cron","source":"docs/jobs-and-cron.md","headings":["Define jobs beside the database","Enqueue atomically from a mutation","Enqueue options","Add cron schedules","Create the background entry","Deploy web, workers, and scheduler","Cloud independent process contract","Delivery and failure semantics","Inspect and operate","Handler security checklist"],"tableOfContents":[{"id":"define-jobs-beside-the-database","title":"Define jobs beside the database","level":2},{"id":"enqueue-atomically-from-a-mutation","title":"Enqueue atomically from a mutation","level":2},{"id":"enqueue-options","title":"Enqueue options","level":3},{"id":"add-cron-schedules","title":"Add cron schedules","level":2},{"id":"create-the-background-entry","title":"Create the background entry","level":2},{"id":"deploy-web-workers-and-scheduler","title":"Deploy web, workers, and scheduler","level":2},{"id":"cloud-independent-process-contract","title":"Cloud-independent process contract","level":2},{"id":"delivery-and-failure-semantics","title":"Delivery and failure semantics","level":2},{"id":"inspect-and-operate","title":"Inspect and operate","level":2},{"id":"handler-security-checklist","title":"Handler security checklist","level":2}],"markdown":"# Durable jobs and cron\n\nClank runs slow, failure-prone, delayed, and scheduled work outside the web request process. The\nqueue is durable in the application's isolated SQLite database, enqueue can share the mutation\ntransaction, and any number of worker processes can cooperate through fenced visibility leases.\nCron uses a separate leased scheduler process and enqueues ordinary jobs; it never executes\nbusiness logic inside the scheduler.\n\nThe delivery contract is **at least once**. A worker can perform an external side effect and crash\nbefore recording success, so handlers must be idempotent or pass a stable idempotency key to the\nexternal system. Clank prevents stale workers from changing queue state, but no local queue can\natomically commit an unrelated remote API's side effect.\n\n## Define jobs beside the database\n\n```ts\nimport {\n  defineDatabase,\n  defineJobs,\n  defineTable,\n  s,\n} from \"@clank.run/framework\";\n\nexport const schema = defineDatabase({\n  reports: defineTable({\n    status: s.enum([\"pending\", \"ready\", \"failed\"]),\n    downloadUrl: s.optional(s.string()),\n  }).owned(),\n});\n\nexport const background = defineJobs({ schema }).jobs(({ job }) => ({\n  reports: {\n    generate: job({\n      args: {\n        reportId: s.id(\"reports\"),\n        format: s.enum([\"csv\", \"json\"]),\n      },\n      queue: \"reports\",\n      priority: 10,\n      timeoutMs: 5 * 60_000,\n      retry: {\n        maxAttempts: 5,\n        initialDelayMs: 1_000,\n        factor: 2,\n        maxDelayMs: 60_000,\n        jitter: 0.2,\n      },\n      description: \"Generate one signed-in user's report outside the request path.\",\n      agent: {\n        title: \"Generate report\",\n        openWorld: false,\n        idempotent: true,\n      },\n      async handler({ db, job, signal }, { reportId, format }) {\n        signal.throwIfAborted();\n        const report = db.read((read) => read.table(\"reports\").get(reportId));\n        if (!report || report.status === \"ready\") return { skipped: true };\n\n        const downloadUrl = await generateReport({\n          reportId,\n          format,\n          idempotencyKey: job.id,\n          signal,\n        });\n        db.transaction((write) => {\n          write.table(\"reports\").patch(reportId, {\n            status: \"ready\",\n            downloadUrl,\n          });\n        });\n        return { skipped: false };\n      },\n    }),\n  },\n}));\n```\n\nArguments are validated before persistence and again before execution. Optional `returns` validates\nand bounds the stored result. Queue names, priorities, timeouts, attempts, delays, payloads, results,\nand error text all have hard limits.\n\nThe handler context contains:\n\n- `db.read()` and `db.transaction()` using the job owner's database scope;\n- immutable job metadata including ID, attempt, schedule, and owner;\n- `signal`, aborted on timeout, cancellation, or lease loss; and\n- `jobs`, a scoped publisher for durable fan-out.\n\nJobs created by a signed-in mutation retain that user's owner scope. A handler cannot use its\ndatabase context to cross an `.owned()` table boundary.\n\n## Enqueue atomically from a mutation\n\nAttach the definition to the backend, then enqueue through the mutation context:\n\n```ts\nimport { defineBackend } from \"@clank.run/framework\";\nimport { background, schema } from \"./background.ts\";\n\nexport const backend = defineBackend({\n  schema,\n  jobs: background,\n}).functions(({ mutation }) => ({\n  reports: {\n    request: mutation({\n      args: { format: s.enum([\"csv\", \"json\"]) },\n      handler: ({ db, jobs }, { format }) => {\n        const reportId = db.table(\"reports\").insert({ status: \"pending\" });\n        const queued = jobs.enqueue(\n          background.jobs.reports.generate,\n          { reportId, format },\n          {\n            idempotencyKey: `generate:${reportId}`,\n            group: `report:${reportId}`,\n          },\n        );\n        return { reportId, jobId: queued.id };\n      },\n    }),\n  },\n}));\n```\n\nThe document insert, global live-data revision, and queue insert use the same `BEGIN IMMEDIATE`.\nThey all commit or all roll back. This is Clank's built-in transactional-outbox path: there is no\ngap where the request commits but its follow-up work disappears.\n\n`enqueue` is synchronous because it only validates and writes local durable state. It never runs\nthe handler in the web process.\n\n### Enqueue options\n\n| Option | Meaning |\n| --- | --- |\n| `runAt` | Earliest epoch-millisecond execution time |\n| `delayMs` | Delay from enqueue; mutually exclusive with `runAt` |\n| `priority` | Higher values are claimed first |\n| `queue` | Route this occurrence to a queue other than the definition default |\n| `idempotencyKey` | Repeated enqueue returns the retained matching job |\n| `group` | Serialize active jobs with the same queue and group |\n\nIdempotency applies while the original terminal job is retained. After an operator or retention\ncleanup purges it, the key can be used again.\n\n## Add cron schedules\n\nSchedules live on a job definition and enqueue the same validated handler:\n\n```ts\nconst background = defineJobs({ schema }).jobs(({ job }) => ({\n  maintenance: {\n    expireInvitations: job({\n      args: { batchSize: s.number({ integer: true, min: 1, max: 1_000 }) },\n      queue: \"maintenance\",\n      schedules: [{\n        name: \"nightly\",\n        cron: \"15 2 * * *\",\n        timezone: \"America/Chicago\",\n        args: { batchSize: 250 },\n        concurrency: \"forbid\",\n        startingDeadlineMs: 30 * 60_000,\n        maxCatchUp: 3,\n      }],\n      handler: expireInvitations,\n    }),\n  },\n}));\n```\n\nClank accepts standard five-field minute, hour, day-of-month, month, and day-of-week expressions.\nLists, ranges, steps, English month/week names, and `@hourly`, `@daily`, `@weekly`, `@monthly`, and\n`@yearly` are supported. Six-field expressions with seconds are rejected. Time zones must be valid\nIANA names supported by the Node runtime.\n\nEach definition has:\n\n- a stable schedule name;\n- `allow`, `forbid`, or `replace` concurrency;\n- a starting deadline for stale occurrences;\n- bounded catch-up after scheduler downtime; and\n- a reversible `suspended` switch.\n\nEvery occurrence gets a deterministic key from the schedule name and scheduled timestamp.\nCompeting schedulers therefore converge on one retained occurrence. `forbid` skips while a prior\noccurrence remains queued, retrying, or running. `replace` cancels the prior occurrence and enqueues\nthe new one. The scheduler itself uses a renewable database lease, so multiple instances are safe.\n\nDay-of-month and day-of-week follow traditional cron OR behavior when both are restricted. Local\ntimes skipped or repeated by daylight-saving transitions follow actual matching UTC minutes; test\nimportant business schedules around both transitions.\n\n## Create the background entry\n\nThe same compiled module serves workers and the scheduler. The process role comes from\n`CLANK_PROCESS_ROLE`:\n\n```ts\nimport { openBackend, runJobProcess } from \"@clank.run/framework\";\nimport { backend } from \"./backend.ts\";\n\nconst runtime = await openBackend(backend, {\n  path: process.env.CLANK_DATABASE_PATH ?? \"app.sqlite\",\n  changePollIntervalMs: 0,\n});\n\nif (!runtime.jobs) throw new Error(\"No jobs are defined.\");\ntry {\n  await runJobProcess(runtime.jobs, {\n    onReady(role, id) {\n      console.log(`${role} ready: ${id}`);\n    },\n  });\n} finally {\n  runtime.close();\n}\n```\n\nRun roles locally in separate terminals:\n\n```sh\nclank jobs worker\nclank jobs worker --concurrency=4 --queues=email,reports\nclank jobs scheduler\n```\n\nThe command reads `clank.deploy.json`, runs its build command, refuses links or an entry outside the\nproject, and launches the compiled jobs entry without a shell.\n\n## Deploy web, workers, and scheduler\n\n```json\n{\n  \"version\": 1,\n  \"entry\": \"dist/server.js\",\n  \"include\": [\"dist\", \"migrations\"],\n  \"build\": {\n    \"command\": [\"clank\", \"build\", \"src\", \"dist\"]\n  },\n  \"database\": {\n    \"path\": \"app.sqlite\",\n    \"migrations\": \"migrations\",\n    \"allowUnsafeMigrations\": false\n  },\n  \"health\": {\n    \"path\": \"/healthz\",\n    \"timeoutMs\": 15000\n  },\n  \"env\": {},\n  \"jobs\": {\n    \"entry\": \"dist/jobs.js\",\n    \"workers\": 2,\n    \"concurrency\": 4,\n    \"queues\": [],\n    \"scheduler\": true\n  }\n}\n```\n\nHosted Clank starts one web process, the requested independent worker processes, and one scheduler.\nIt health-checks the web process and checks that every background process survives startup before\nactivation. An unexpected background exit marks the release crashed and enters bounded automatic\nrestart. Logs identify `worker[n]` and `scheduler` streams. Memory diagnostics attribute every role\nseparately.\n\nFor a code-only rolling update, Clank keeps the old web process serving while it gracefully\nquiesces its workers and scheduler. It then starts the candidate process group and switches ingress\nonly after startup succeeds. If the candidate fails, the prior background set is resumed. This\navoids running old and new handler code at the same time.\n\nPending migrations still use an exclusive maintenance path so no web or background process holds\nthe database during migration or restore.\n\n## Cloud-independent process contract\n\nClank does not depend on Railway job primitives. Any process manager can run:\n\n```text\nweb        node dist/server.js\nworker     CLANK_PROCESS_ROLE=worker node dist/jobs.js\nscheduler  CLANK_PROCESS_ROLE=scheduler node dist/jobs.js\n```\n\nWorker replicas may set `CLANK_WORKER_CONCURRENCY` and a comma-separated\n`CLANK_WORKER_QUEUES`. SIGTERM and SIGINT stop claims and wait for in-flight handlers before exit.\nThis works with Railway, a systemd host, Docker Compose, Kubernetes, Fly.io, Render, or another\nprovider that can give the process group access to the same durable application database.\n\nThe included driver coordinates through SQLite and therefore requires every role for one app to\nshare the same durable POSIX database files. Independent nodes with private ephemeral disks are not\na shared queue. On those platforms, keep the Clank supervisor and its per-app processes on the\nvolume-owning host, or implement the same lease/idempotency contract against a transactional shared\nstore before scaling across hosts.\n\n## Delivery and failure semantics\n\n1. A worker atomically claims one due job and records a random lease token, owner, expiry, and\n   incremented attempt.\n2. It renews the visibility lease while the handler runs.\n3. Completion compares the token and worker identity. A stale worker cannot settle reclaimed work.\n4. A crash leaves the job running until lease expiry. Another worker reclaims it and applies\n   bounded exponential backoff.\n5. Exhausted work moves to `dead`; it is never silently discarded.\n6. Cancellation fences success, aborts a live handler on its next heartbeat, and leaves an explicit\n   `cancelled` record.\n\nThis follows the same visibility-timeout pattern documented for\n[Amazon SQS](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html).\nCron behavior intentionally uses the familiar five-field, deadline, concurrency, and time-zone\nconcepts documented for [Kubernetes CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/).\nSQLite access uses Node's built-in\n[`node:sqlite`](https://nodejs.org/api/sqlite.html), so the framework adds no package dependency.\n\n## Inspect and operate\n\n`runtime.jobs` exposes:\n\n```ts\nruntime.jobs.stats();\nruntime.jobs.list({ state: \"dead\", queue: \"email\", limit: 100 });\nruntime.jobs.get(jobId);\nruntime.jobs.events(jobId, { limit: 100 });\nruntime.jobs.cancel(jobId);\nruntime.jobs.retry(jobId, { runAt: Date.now() });\nruntime.jobs.purge({\n  states: [\"succeeded\", \"cancelled\"],\n  before: Date.now() - 7 * 24 * 60 * 60_000,\n  limit: 1_000,\n});\n```\n\nSuccessful and cancelled history is retained for seven days by default and cleaned in bounded\nbatches. Dead letters are retained indefinitely by default. Override `openBackend(..., { jobs })`\nor `openJobs(..., { retention })` when an application's compliance rules differ. Events are\nglobally bounded and are deleted with their job.\n\nQueue stats include every state, currently due work, and the oldest due timestamp. The standard\napplication manifest at `/__clank/manifest` also includes job argument schemas, queues, retry\npolicy, schedules, descriptions, and agent metadata. A manifest entry documents a background\ncapability; it does not make that job remotely callable. Expose an authenticated mutation when a\nperson or agent should be able to request it.\n\nFor telemetry, use stable job name, queue, state, and attempt fields. Do not use job IDs, user IDs,\narguments, or error messages as metric labels. The\n[OpenTelemetry messaging conventions](https://opentelemetry.io/docs/specs/semconv/messaging/messaging-spans/)\nare a useful interoperability target for traces.\n\n## Handler security checklist\n\n- Treat stored arguments as untrusted even though Clank revalidates them.\n- Make external writes idempotent using `context.job.id` or a domain key.\n- Pass `context.signal` into `fetch`, storage, and other cancellable work.\n- Set an explicit timeout shorter than the provider's termination deadline.\n- Keep retryable and permanent failures distinguishable in error reporting.\n- Never put credentials or private payloads in queue names, group keys, idempotency keys, logs, or\n  agent metadata.\n- Bound fan-out and avoid a handler that can enqueue itself without a terminating condition.\n- Restrict worker queue allowlists when a process has privileged network or secret access.\n- Inspect and alert on dead jobs and oldest-due age.\n- Back up the application database; queue state is part of that same database and restore point.\n\n"}