{"protocol":"clank-doc/1","frameworkVersion":"0.9.4","slug":"provider-data-lifecycle","title":"Provider data lifecycle","description":"Remote application placement is more than copying a release. A trusted runtime provider must keep each project's SQLite file outside immutable code, migrate it before traffic changes, retain a known rollback point, recover interrupted work,","group":{"id":"deploy","title":"Deploy and operate"},"url":"https://docs.clank.run/docs/provider-data-lifecycle","source":"docs/provider-data-lifecycle.md","headings":["What apply guarantees","Database modes","Retry and crash behavior","Snapshot, rollback, and deletion","Runtime launch and remaining hosted activation boundary"],"tableOfContents":[{"id":"what-apply-guarantees","title":"What `apply` guarantees","level":2},{"id":"database-modes","title":"Database modes","level":3},{"id":"retry-and-crash-behavior","title":"Retry and crash behavior","level":2},{"id":"snapshot-rollback-and-deletion","title":"Snapshot, rollback, and deletion","level":2},{"id":"runtime-launch-and-remaining-hosted-activation-boundary","title":"Runtime launch and remaining hosted activation boundary","level":2}],"markdown":"# Provider data lifecycle\n\nRemote application placement is more than copying a release. A trusted runtime provider must keep\neach project's SQLite file outside immutable code, migrate it before traffic changes, retain a\nknown rollback point, recover interrupted work, and erase it only after an explicit destructive\nrequest.\n\nClank packages that boundary as `@clank.run/framework/provider-data`. It uses Node's built-in\nSQLite and filesystem APIs and has no provider SDK or runtime dependency.\n\n```ts\nimport {\n  openDeploymentProviderDataStore,\n} from \"@clank.run/framework/provider-data\";\nimport type {\n  DeploymentProvider,\n} from \"@clank.run/framework/provider\";\n\nconst data = await openDeploymentProviderDataStore({\n  rootDirectory: \"/var/lib/clank-provider\",\n  maxDatabaseBytes: 1024 * 1024 * 1024,\n});\n\nconst provider: DeploymentProvider = {\n  kind: \"microvm\",\n  async reconcile(request) {\n    if (request.desired.state === \"stopped\") {\n      await stopProject(request.operation.projectId, request.signal);\n      return;\n    }\n    if (!request.runtime) {\n      throw new Error(\"This provider requires clank-runtime/1.\");\n    }\n\n    // SQLite migrations and restore are stopped-runtime operations. Drain and\n    // stop the current generation before allowing the store to mutate data.\n    await stopProject(request.operation.projectId, request.signal);\n\n    try {\n      await data.apply({\n        operation: request.operation,\n        desired: request.desired,\n        runtime: request.runtime,\n        signal: request.signal,\n      }, async (prepared) => {\n        // The release and migrated database are private at this point.\n        const candidate = await startPrivateCandidate({\n          releaseDirectory: prepared.releaseDirectory,\n          databasePath: prepared.databasePath,\n          environment: prepared.environment,\n          signal: request.signal,\n        });\n        try {\n          await candidate.checkHealth();\n          await retainPrivateCandidate(candidate, prepared);\n        } catch (error) {\n          await candidate.stop();\n          throw error;\n        }\n\n        // Do not publish ingress here. Store enough provider-local runtime state\n        // to make the same capsule/fence retry idempotent, then let apply commit.\n      });\n    } catch (error) {\n      await restartCommittedProject(await data.inspect(request.operation.projectId));\n      throw error;\n    }\n\n    // Only a committed generation may be switched into managed ingress.\n    await publishProjectRoute(request.operation.projectId, request.signal);\n  },\n};\n```\n\nThe callback receives final environment values and the managed-ingress token in memory. Treat it as\na secret-bearing runtime boundary: do not log, serialize, persist, or return the prepared object.\nIt also receives the exact normalized deployment config from the verified capsule, so a launcher\ndoes not need a second caller-supplied copy.\nIt may start an unreachable candidate and run its health check, but it must not expose that\ncandidate to public traffic. If the callback throws or the signal is aborted, Clank restores the\nprior database and removes the candidate release. Provider code must also stop a failed candidate\nand restart the prior committed release before restoring ingress.\n\n`apply`, `rollback`, and `delete` are stopped-runtime operations. The store owns files, not\nprocesses, so it cannot prove that an application has released SQLite handles. Drain writes and\nstop every web/job/scheduler process for the project before calling them. A live\n`snapshot(projectId)` is safe because it uses SQLite's online backup API.\n\n## What `apply` guarantees\n\nThe store independently rehashes and decodes the exact `clank-runtime/1` capsule. It then binds the\ncapsule to the provider operation's project and the desired state's protocol, release, and\ngeneration. Calling the store directly cannot bypass the provider transport checks.\n\nFor one accepted generation it:\n\n1. rejects stale generations and monotonically stale operation fences;\n2. extracts the verified release into an owner-only staging directory;\n3. creates a transactionally consistent safety copy of existing SQLite data;\n4. initializes, preserves, or replaces the project database according to the capsule;\n5. applies ordered immutable migrations from that exact release;\n6. enforces the configured resulting database-size ceiling;\n7. promotes the immutable release and calls the private validation hook;\n8. atomically commits generation metadata and its fence high-water mark; and\n9. retains only the active release, its immediate predecessor, and the snapshot needed to undo the\n   active generation.\n\nThe provider root uses this layout:\n\n```text\n/var/lib/clank-provider/\n└── projects/\n    └── <project-id>/\n        ├── data/          # the project's SQLite file\n        ├── generations/   # active and one rollback release\n        ├── recovery/      # immediate rollback snapshot\n        ├── .staging/      # unreachable in-progress extraction\n        ├── state.json     # active/previous generation, digest, and paths\n        ├── fence.json     # durable stale-operation high-water mark\n        └── journal.json   # exists only while apply or rollback is recoverable\n```\n\nDirectories are `0700`; metadata, snapshots, and databases are `0600`. Metadata is strictly\nversioned, size-bounded, exact-field decoded, project-bound, and path-scoped. Symbolic-link and\nunsafe-permission storage paths fail closed. Environment values and ingress tokens never enter\nthese files. Start a provider entry point with `process.umask(0o077)`, as the packaged\n`clank-runner` does. Database limits include an existing SQLite file plus its WAL and shared-memory\nsidecars, so a large uncheckpointed journal cannot bypass the configured disk boundary.\n\n### Database modes\n\n| Mode | Snapshot | Behavior |\n| --- | --- | --- |\n| `initialize` | optional | Requires no committed generation or existing database. An optional seed snapshot is restored before migrations. |\n| `preserve` | forbidden | Requires the currently committed database and migrates it in place behind a safety snapshot. |\n| `replace` | required | Restores the supplied integrity-checked SQLite snapshot, then applies the release's migrations. |\n\nThe database path is bound to the release artifact and cannot change between committed\ngenerations. Moving a database is an explicit export/delete/import operation, not a side effect of\na routine release. A database file found without committed provider metadata is never adopted or\noverwritten implicitly; inspect it through an operator recovery workflow or remove it through the\nconfirmed project deletion path.\n\n## Retry and crash behavior\n\n`apply` serializes work per project in one provider process. The durable generation and fence make\nexact retries idempotent: a repeated committed capsule runs the private validation callback with\n`alreadyCommitted: true` and does not replay migrations. Before that callback, Clank walks the\nstored immutable release and rechecks its exact file set, owners, modes, sizes, and SHA-256 values\nagainst the capsule. A different capsule for the same generation or a changed stored release fails\nas a conflict.\n\nIf validation starts a private candidate, pass the optional discard callback:\n\n```ts\nawait data.apply(input, async (prepared) => {\n  candidate = await runtimes.launch({\n    prepared,\n    signal: input.signal,\n    deferBackground: true,\n  });\n}, async () => {\n  if (candidate) {\n    await runtimes.stop(candidate.projectId, candidate.generation);\n  }\n});\n```\n\nClank invokes discard before restoring an uncommitted database whenever the prepared candidate\nwas exposed to validation. If cleanup fails, the database stays at the journaled intermediate\nstate and the operation fails closed. After the launcher removes the process, a later\n`inspect`, `apply`, or snapshot safely completes journal recovery. Background workers should be\ndeferred until after apply commits; the complete provider service enforces both rules.\n\nThe journal makes the filesystem transaction restart-safe:\n\n- before metadata commits, recovery restores the safety snapshot and removes the candidate;\n- after metadata commits, recovery keeps the new database and finishes cleanup;\n- an interrupted operator rollback restores the pre-rollback database while old metadata is\n  active; and\n- an already committed rollback finishes deleting only the generation it replaced.\n\n`inspect`, `snapshot`, `rollback`, and the next `apply` recover an interrupted transaction before\nreturning. They also remove unreferenced staging, generation, and recovery entries. The state\nrename is the commit point; a cleanup error can never roll database bytes back underneath new\nmetadata.\n\nUse one writer process for a provider root. The in-process project mutex is not a distributed\nfilesystem lock. A stateful project must remain pinned to the provider node that owns its data, and\nrolling provider replacement must stop the old writer before mounting that root in the new one.\nControl-plane placement and ingress activation enforce that topology when the hosted integration\nis enabled.\n\n## Snapshot, rollback, and deletion\n\n`snapshot(projectId)` uses SQLite's backup API, verifies and bounds the result, and returns bytes\nplus their SHA-256, project, release, and generation binding. Send those bytes directly into an\nencrypted [recovery repository](recovery.md); do not treat the immediate rollback snapshot as an\nindependent disaster-recovery backup.\n\n```ts\nconst snapshot = await data.snapshot(\"orbit_tasks\");\n\nawait data.rollback({\n  projectId: \"orbit_tasks\",\n  generation: 42,\n  confirmation: \"rollback orbit_tasks 42\",\n});\n\nawait data.delete({\n  projectId: \"orbit_tasks\",\n  confirmation: \"delete orbit_tasks\",\n});\n```\n\nRollback restores only the immediately preceding database/release and consumes that rollback\npoint. It requires the exact current generation and confirmation string, carries the latest fence\nforward, and is itself journaled. Deletion recursively removes only the provider-owned project\nroot and requires an exact confirmation. Stop and revoke ingress before either operation; restore\nand deletion do not coordinate public traffic themselves.\n\n## Runtime launch and remaining hosted activation boundary\n\nThis package completes provider-side release staging, per-project SQLite\ninitialize/preserve/replace, migrations, consistent export, one-generation rollback, crash\nrecovery, and confirmed deletion. It deliberately does not launch untrusted code, publish routes,\nissue TLS certificates, replicate recovery backups, or move a stateful project between nodes.\n\nThe package-supported [Provider Docker runtime](provider-docker-runtime.md) now binds this prepared\nstate to resource-bounded web, worker, and scheduler containers, private loopback health,\nexact-owner orphan cleanup, and fail-closed restart reconciliation. The built-in control plane\nstill uses its local supervisor. The [complete deployment provider\nservice](provider-service.md) binds provider data, deferred Docker activation, durable\ngeneration/fence intent, and private ingress end to end. Remote placement remains disabled until\nthe next integration binds that service to:\n\n- atomic ingress publish/revoke with generation and token checks;\n- durable provider-node pinning and rolling-update behavior;\n- encrypted backup replication and restore drills; and\n- end-to-end lease-loss, retry-after-commit, deletion, and traffic-switch tests.\n\nContinue with [Remote runtime placement](runtime-placement.md), [Deployment provider\nadapters](provider-adapters.md), [Provider runtime ingress](provider-runtime-ingress.md),\n[Provider Docker runtime](provider-docker-runtime.md),\n[Complete deployment provider service](provider-service.md),\n[Managed ingress](data-plane.md), and [Backup and disaster recovery](recovery.md).\n"}