Blixt Documentation v2.9

Cloudflare R2

Create an R2 bucket, and set up the queue that carries its change notifications.

Create a bucket

  1. Sign in to the Cloudflare dashboard.
  2. Go to R2 Object Storage and click Create bucket.
  3. Give it a name and choose a location (or a jurisdiction, if your data must stay in the EU).
  4. Copy your Account ID from the R2 overview page. BlixtFS uses it to build the S3 endpoint.

Change notifications

R2 has no S3 notification API. A bucket’s object events are routed by a notification rule to a Cloudflare Queue, and BlixtFS drains that queue over the Queues HTTP pull endpoint. Those calls go to api.cloudflare.com on a Cloudflare API token — a different credential from the R2 access keys, which sign S3 requests only.

Two properties of R2 queues shape how this behaves, and both differ from other object stores (S3, GCS, and so on):

  • One notification queue per bucket. R2 rejects overlapping notification rules, so all of a bucket’s events go to exactly one queue. There is no per-server subscription to create, and pointing a second BlixtFS server at the same bucket adds no second stream.
  • Every notification is delivered once. Consumers of a queue compete for its messages: whichever consumer pulls a message is the only one that sees it. A BlixtFS deployment consumes the queue from a single place — the notifier or the updater runs one replica, and the embedded path runs in the indexing server — so a deployment sees every event exactly once. Two separate deployments pointed at the same R2 bucket would split the stream between them, each seeing roughly half the events — silently, because a missing notification looks exactly like a file nobody touched. Run one deployment per bucket, or turn on the event log below.

BlixtFS discovers the queue rather than assuming it: it asks R2 which queue the bucket’s events reach and binds to that one, so a bucket repointed at a different queue keeps working instead of quietly polling an empty queue. CLOUDFLARE_QUEUE names the queue to use only when BlixtFS has to create a rule for a bucket that has none.

Create the queue, give it an HTTP pull consumer, and route the bucket’s events to it. BlixtFS logs these same commands for any bucket it finds unconfigured:

wrangler queues create bfs-queue
wrangler queues consumer http add bfs-queue
wrangler r2 bucket notification create my-bucket --queue bfs-queue \
  --event-type object-create --event-type object-delete

For a bucket in a jurisdiction, add --jurisdiction eu (or fedramp) to the queue and notification commands: a jurisdiction is a separate namespace, and commands that omit it address the default one and appear to succeed. BlixtFS creates a missing notification rule itself unless rule management is turned off; the queue and its pull consumer are always yours to create.

Several regions on one bucket: the event log

The one-queue-one-consumer limit above is a property of the queue, not of BlixtFS. The event log works around it: one server drains the queue and writes the events as small batched objects in the bucket, under a reserved .__bfs__/eventlog/ prefix. Every other server simply lists that prefix. Listing takes nothing away, so any number of regions can read the same log, and adding a region means starting a server — nothing else in the deployment learns about it.

The credential difference is the reason to use it. Draining the queue needs a Cloudflare API token, which can create and delete queues and rewrite notification rules across the whole account. Reading the log needs only the R2 access key the server already has. So every region except the one that relays holds strictly less — and in Kubernetes that relay is the notifier, a deployment that exists for nothing else and runs a single replica.

Turn it on with --event_log, and run one notifier to drain the queue. In the Helm chart, set eventLog.enabled and notifier.enabled together — the log needs a writer, so a notifier with the log switched off refuses to start. With Kustomize, add the components/eventlog component to your overlay. Log objects are reclaimed automatically after --event_log_retention_minutes (an hour by default): the log is a cache in front of the filesystem scan rather than an archive, so nothing is lost when an entry expires, and a server offline longer than that catches up with a scan instead.

You can use several notifiers for high availability, for example in separate regions or using separate cloud providers.

Without an API token, R2 serves normally but delivers no notifications. Everything written through BlixtFS is visible immediately; changes made to the bucket outside BlixtFS — from the Cloudflare dashboard, wrangler or another S3 client — are picked up by the next filesystem scan instead of straight away.

Credentials and permissions

  1. In the Cloudflare dashboard, go to R2 Object Storage → API → Manage API tokens.
  2. Create an Account API token with Object Read & Write permission for your bucket.
  3. Copy the Access Key ID and Secret Access Key. These are S3-compatible credentials.
  4. Enter these, along with your account ID, in BlixtFS during setup.
  5. For change notifications, also create a user API token with Account:Queues (Edit) and Account:Workers R2 Storage (Edit). This is a separate credential: the Cloudflare API accepts no S3 key.

Environment variables

R2 configuration does not use a credentials file. Account ID and credentials are supplied as environment variables, or command-line flags.

Required to serve a bucket

  • CLOUDFLARE_ACCOUNT_ID (--cloudflare_account_id) — the R2 account ID. The S3 endpoint is derived from it.
  • CLOUDFLARE_ACCESS_KEY (--cloudflare_access_key) — the R2 API token’s Access Key ID.
  • CLOUDFLARE_SECRET_KEY (--cloudflare_secret_key) — the R2 API token’s Secret Access Key.

Required for change notifications

Notifications arrive over the Cloudflare Queues API, which accepts no S3 credential, so this is a second credential rather than the keys above. The R2 access key ID is a token’s id and the secret key is the SHA-256 of the token value, so the S3 pair cannot be turned back into anything api.cloudflare.com will take. Set the token, and the R2 bucket stays in step with changes made outside BlixtFS.

  • CLOUDFLARE_API_TOKEN (--cloudflare_api_token) — an API token with Account:Queues (Edit) and Account:Workers R2 Storage (Edit). Leave it unset and R2 serves normally, with out-of-band changes picked up by the next scan.
  • CLOUDFLARE_QUEUE (--cloudflare_queue) — the queue to route a bucket’s events to when BlixtFS has to create a notification rule. An existing rule’s queue is discovered, not taken from here.

Optional

  • CLOUDFLARE_JURISDICTION (--cloudflare_jurisdiction) — eu or fedramp. A bucket created in a jurisdiction is only reachable through that jurisdiction’s endpoint.
  • CLOUDFLARE_BUCKETS (CLOUDFLARE_BUCKET, or R2_BUCKETS / R2_BUCKET) — buckets to serve, instead of listing them as arguments. A bucket may carry attributes after a colon, as it can on the command line: r2://my-bucket:readonly,lazy. The attributes are readonly (or ro), anonymous, lazy, pubsub, nopubsub, rescan= (a duration, or never) and topic= (an existing notification topic to subscribe to, GCS and AWS only); anything else stops BlixtFS from starting rather than being ignored. See Read-only buckets for what readonly and anonymous guarantee.

Summary

export CLOUDFLARE_ACCOUNT_ID=your-account-id
export CLOUDFLARE_ACCESS_KEY=your-access-key-id
export CLOUDFLARE_SECRET_KEY=your-secret-access-key

# Optional: change notifications, over a Cloudflare Queue.
export CLOUDFLARE_API_TOKEN=your-api-token
export CLOUDFLARE_QUEUE=bfs-queue

In Docker, pass them with -e; in Kubernetes, the Helm chart reads them from the bfs-cloudflare Secret. Keep the secret key and the API token out of shell history and out of version control.