{"protocol":"clank-doc/1","frameworkVersion":"0.9.4","slug":"object-storage","title":"Object storage","description":"Clank exposes one small object contract for application files, release archives, and encrypted backup envelopes. The framework includes an atomic local implementation and a zero dependency S3 compatible implementation. Local storage remains","group":{"id":"full-stack","title":"Full stack"},"url":"https://docs.clank.run/docs/object-storage","source":"docs/object-storage.md","headings":["Contract","Atomic local storage","S3 compatible storage","Control plane release uploads","Encrypted recovery backups","Railway without surprise cost","Security and operations"],"tableOfContents":[{"id":"contract","title":"Contract","level":2},{"id":"atomic-local-storage","title":"Atomic local storage","level":2},{"id":"s3-compatible-storage","title":"S3-compatible storage","level":2},{"id":"control-plane-release-uploads","title":"Control-plane release uploads","level":2},{"id":"encrypted-recovery-backups","title":"Encrypted recovery backups","level":2},{"id":"railway-without-surprise-cost","title":"Railway without surprise cost","level":2},{"id":"security-and-operations","title":"Security and operations","level":2}],"markdown":"# Object storage\n\nClank exposes one small object contract for application files, release archives, and encrypted\nbackup envelopes. The framework includes an atomic local implementation and a zero-dependency\nS3-compatible implementation. Local storage remains the development and single-host default; S3\ncompatibility is the portability boundary for independent compute nodes.\n\n## Contract\n\n```ts\nimport type { ObjectStore } from \"@clank.run/framework/object-storage\";\n\nasync function retainRelease(\n  objects: ObjectStore,\n  projectId: string,\n  releaseId: string,\n  bytes: Uint8Array,\n) {\n  return objects.put(\n    `releases/${projectId}/${releaseId}.clank.gz`,\n    bytes,\n    { contentType: \"application/vnd.clank.deploy+gzip\" },\n  );\n}\n```\n\n`ObjectStore` has four operations:\n\n| Operation | Result |\n| --- | --- |\n| `put(key, bytes, { contentType? })` | Immutable metadata for the completed replacement |\n| `get(key)` | Verified metadata and a copied byte array, or `null` |\n| `stat(key)` | Validated metadata without downloading the object, or `null` |\n| `delete(key)` | Whether a current object was removed |\n\nMetadata includes the logical key, size, SHA-256, normalized media type, creation time, and update\ntime. Logical keys use bounded portable segments; they are never interpolated as raw filesystem\npaths or unsigned URLs. `put` snapshots a mutable input before returning.\n\nThe current contract deliberately buffers one bounded object. It is intended for release archives,\nbounded encrypted-backup chunks, and ordinary application files up to 1 GiB—not multi-gigabyte\nmedia ingest as one object. The backup manager splits a large encrypted envelope into independently\nverified chunks before it crosses this boundary.\nMultipart upload, public bucket administration, CDN policy, browser presigning, and lifecycle rules\nremain explicit provider integrations.\n\n## Atomic local storage\n\n```ts\nimport { openLocalObjectStore } from \"@clank.run/framework/object-storage\";\n\nconst objects = await openLocalObjectStore({\n  directory: \".data/objects\",\n  maxObjectBytes: 100 * 1024 * 1024,\n});\n```\n\nEach object is stored as one owner-only envelope containing integrity metadata and bytes.\nUpdates write a new same-directory temporary file and replace the prior generation atomically.\nReads use a no-follow descriptor and compare the opened file with its current path, type, owner,\nmode, and size. `get` then checks the exact byte length and SHA-256.\n\nLocal object storage is one-host durability. It does not protect against volume or region loss.\n\n## S3-compatible storage\n\n```ts\nimport { createS3ObjectStore } from \"@clank.run/framework/object-storage\";\n\nconst objects = createS3ObjectStore({\n  endpoint: process.env.AWS_ENDPOINT_URL!,\n  region: process.env.AWS_DEFAULT_REGION ?? \"auto\",\n  bucket: process.env.AWS_S3_BUCKET_NAME!,\n  accessKeyId: process.env.AWS_ACCESS_KEY_ID!,\n  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,\n  sessionToken: process.env.AWS_SESSION_TOKEN,\n  prefix: \"clank-production\",\n  maxObjectBytes: 100 * 1024 * 1024,\n});\n```\n\nThe adapter implements the portable `HEAD`, `GET`, `PUT`, and `DELETE` object subset through native\n`fetch`. It works with virtual-hosted endpoints by default. Set `pathStyle: true` for a loopback\nMinIO-compatible server or an older endpoint that requires bucket names in the URL path. Dotted\nbucket names require path style so TLS hostname matching is unambiguous.\n\nEvery request:\n\n- requires HTTPS except for explicit loopback development;\n- refuses endpoint credentials, query strings, fragments, and redirects;\n- uses AWS Signature Version 4 with the exact single-chunk SHA-256;\n- signs the host, payload hash, date, session token, content type, and Clank integrity metadata;\n- retries only the identical idempotent request after network, `429`, or `5xx` failures;\n- applies a deadline across response-body consumption;\n- bounds provider error bodies before discarding them; and\n- returns stable public error codes without provider XML, bucket names, object keys, or credentials.\n\n`get` does not treat an ETag as a content digest. It requires Clank's stored SHA-256, size, media\ntype, and timestamps, bounds the response independently, and hashes the bytes again. Objects\ncreated outside Clank without that metadata fail closed.\n\nThe signing behavior follows the official [AWS Signature Version 4 single-chunk\ncontract](https://docs.aws.amazon.com/AmazonS3/latest/developerguide/sig-v4-header-based-auth.html).\nRailway Buckets expose S3-compatible credentials and virtual-hosted URLs; the\n[Railway bucket reference](https://docs.railway.com/storage-buckets) documents its current variable\nnames and billing. Cloudflare publishes the supported subset for\n[R2's S3 API](https://developers.cloudflare.com/r2/api/s3/api/). MinIO and other self-hosted\nimplementations can use the same contract.\n\n## Control-plane release uploads\n\nWhen authenticated remote deployment nodes are enabled, `clank-platform` retains every new\noriginal upload so the exact leased release can be downloaded by its assigned node. Local\nowner-only files remain the default. Select object storage explicitly:\n\n```sh\nexport CLANK_RUNNER_REGISTRATION_TOKEN=\"$(your-secret-manager read runner-enrollment)\"\nexport CLANK_RUNNER_ARTIFACT_STORE=s3\nexport CLANK_RUNNER_ARTIFACT_NAMESPACE=production-releases-v1\nexport CLANK_OBJECT_ENDPOINT=https://objects.example.com\nexport CLANK_OBJECT_REGION=auto\nexport CLANK_OBJECT_BUCKET=clank-releases\nexport CLANK_OBJECT_ACCESS_KEY_ID=\"$(your-secret-manager read object-access-key)\"\nexport CLANK_OBJECT_SECRET_ACCESS_KEY=\"$(your-secret-manager read object-secret-key)\"\nexport CLANK_OBJECT_PREFIX=installation-01\n```\n\nStandard `AWS_ENDPOINT_URL`, `AWS_DEFAULT_REGION`, `AWS_S3_BUCKET_NAME`,\n`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` variables are fallback names,\nincluding Railway Bucket references. Explicit `CLANK_OBJECT_*` values take precedence.\n\nEach release persists:\n\n- the operator-selected repository namespace;\n- an exact `runner-artifacts/<project>/<release>/<sha256>.clank.gz` key;\n- the original byte length and SHA-256; and\n- whether its runner copy is local or object-backed in the release API.\n\nThe platform checks the metadata returned by `put`, rehashes each leased `get`, and removes the\nexact object during release cleanup and site deletion. A failed or ambiguous write is cleaned\nbefore its quota reservation is released. A namespace or key mismatch refuses reads and deletion;\nrestore the original repository configuration rather than reusing an identity for a different\nbucket. Existing local releases never move implicitly.\n\n## Encrypted recovery backups\n\nThe framework and deployment platform can store completed encrypted recovery points through the\nsame object contract:\n\n```ts\nimport { createS3ObjectStore } from \"@clank.run/framework/object-storage\";\nimport { openBackupManager } from \"@clank.run/framework/recovery\";\n\nconst store = createS3ObjectStore({\n  endpoint: process.env.AWS_ENDPOINT_URL!,\n  region: process.env.AWS_DEFAULT_REGION ?? \"auto\",\n  bucket: process.env.AWS_S3_BUCKET_NAME!,\n  accessKeyId: process.env.AWS_ACCESS_KEY_ID!,\n  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,\n  maxObjectBytes: 32 * 1024 * 1024,\n});\n\nconst backups = await openBackupManager({\n  databasePath: \"app.sqlite\",\n  repositoryDirectory: \".data/recovery\",\n  encryptionKey: process.env.BACKUP_KEY!,\n  objects: {\n    store,\n    namespace: \"production-recovery-v1\",\n    repositoryId: \"orbit-tasks\",\n    chunkBytes: 8 * 1024 * 1024,\n  },\n});\n```\n\nThe local directory remains an owner-only staging and compatibility area. The manager:\n\n1. creates a consistent SQLite snapshot and AES-256-GCM envelope locally;\n2. records an authenticated `uploading` catalog entry before uploading deterministic chunk keys;\n3. verifies every returned key, size, media type, and SHA-256;\n4. downloads, reassembles, decrypts, hashes, and runs SQLite `integrity_check` when verification is\n   enabled;\n5. publishes the catalog entry as `active`; and\n6. removes the completed local copy only after the remote recovery point is usable.\n\nA failed upload leaves the encrypted local copy available and retries its promotion before the next\nnew backup. Existing local backups are promoted the same way. Retention spans both locations while\nthat transition is in progress. Restore never trusts provider metadata alone: it checks the\ncatalog HMAC, per-chunk digest, AES-GCM tag, plaintext digest and length, then SQLite integrity.\n\nThe namespace is the durable identity of the physical repository. Do not reuse it for a different\nbucket. `repositoryId` isolates one database's catalog and keys; the platform supplies its project\nID automatically. A prefix separates installations, but provider credentials and bucket policy\nremain the authorization boundary.\n\nFor `clank-platform`, select the backend explicitly:\n\n```sh\nexport CLANK_BACKUP_STORE=s3\nexport CLANK_BACKUP_NAMESPACE=production-recovery-v1\nexport CLANK_BACKUP_PREFIX=backups\nexport CLANK_BACKUP_CHUNK_BYTES=8388608\n```\n\nThese settings use the same `CLANK_OBJECT_*` or fallback `AWS_*` connection variables shown above.\nThe control database binds the namespace and backup root on first use. Startup fails if either is\nremoved or changed, preventing a typo from presenting an empty backup history. Move or copy all\nkeys first and preserve the identity when intentionally changing provider credentials or\nendpoints.\n\n## Railway without surprise cost\n\nCreating this adapter does not create a bucket. Provision one only when its failure-domain benefit\nis worth the added resource:\n\n```sh\nrailway bucket create\nrailway bucket credentials --bucket <name>\n```\n\nUse Railway variable references so credentials stay out of Git and local shell history. Give each\nenvironment a separate bucket or at least a unique high-entropy prefix. Use an IAM/bucket policy\nlimited to the required bucket and prefix when the provider supports it.\n\nAs of July 2026, Railway documents bucket storage at `$0.015/GB-month`, with bucket egress and S3\noperations included. Service-to-bucket traffic can still count as service egress, so monitor both\nthe bucket and service. The local adapter costs no additional managed resource and remains the\nright default for development.\n\n## Security and operations\n\n- Keep access keys only in the control plane or trusted worker environment, never in application\n  HTML, operation payloads, logs, or release archives.\n- Use temporary credentials and `sessionToken` where the provider supports them.\n- Separate production, preview, and test namespaces. A prefix is a routing boundary only when the\n  provider policy also restricts credentials to it.\n- Encrypt sensitive data before upload. Clank recovery backups are already AES-256-GCM envelopes;\n  release archives intentionally exclude managed secrets and application databases.\n- Alert on denied requests, repeated integrity failures, timeouts, and storage growth.\n- Exercise restore and provider-outage drills. Remote existence is not proof that an object can be\n  decrypted, verified, and used.\n\nRuntime directories, live databases, and migration rollback snapshots remain local. New\nremote-runner uploads and encrypted recovery backups can independently opt into the object\ncontract; both remain local by default. Provider-backed storage is not a substitute for control\ndatabase and key recovery.\n"}