Change notifications
How BlixtFS learns about objects created, replaced or deleted by programs that write to the bucket directly.
Why notifications matter
Anything written through BlixtFS is visible to every BlixtFS client immediately. But the bucket is still an ordinary bucket: pipelines, other applications and people using the cloud provider’s console can modify objects directly. Change notifications are how BlixtFS finds out about those modifications.
How it works
- When BlixtFS starts serving a bucket, it sets up (or connects to) the provider’s event mechanism for that bucket.
- The provider publishes an event whenever an object is created, replaced, deleted, or metadata is changed.
- The updater receives the event, persists the event and hands notifies a file server.
- The file server compares the event’s generation with the index. Newer events update the file. Stale ones, and echoes of BlixtFS’s own writes, are dropped.
Changes typically appear in the filesystem within seconds.
Provider mechanisms
| Provider | Mechanism | Set up by |
|---|---|---|
| Google Cloud Storage | Pub/Sub | BlixtFS (one IAM grant needed, see below) |
| Amazon S3 | S3 events → SNS → SQS | BlixtFS |
| Azure Blob Storage | Event Grid → Service Bus | BlixtFS |
| Oracle OCI | Events → Streaming | BlixtFS |
| MinIO | Bucket notification webhook | BlixtFS |
| Cloudflare R2 | Cloudflare Queues | You create the queue; BlixtFS creates the rule. Needs an API token |
| CoreWeave | None | Changes are found by consistency checks |
For Google Cloud Storage, the storage service account needs permission to
publish to the blixtfs Pub/Sub topic. The
setup guide has the exact
commands. Cloudflare R2 setup is described in the
guide.
Without notifications
Some buckets can’t deliver notifications. The provider may not support them, your credentials may not allow setting them up, or the bucket may be served read-only. BlixtFS still serves such buckets. Changes made outside BlixtFS are picked up by:
- the periodic consistency check, and
- manual fsck requests.
The event log
Some providers deliver each event to only one consumer, and setting up a
subscription usually requires broad account permissions. The event log
solves both problems. One notifier drains
the bucket’s notifications and writes them, in small batches, into the bucket
itself under a reserved .__bfs__/eventlog/ prefix. Every other deployment
(in any region, or any cloud) reads changes by listing that prefix, using only
the storage credentials it already has.
Log entries expire after a retention period, an hour by default. A server that has been offline for longer catches up with a full rescan instead, so nothing is lost.
Turn it on with --event_log, and run exactly one notifier per bucket.
Seeing changes promptly on NFS clients
NFS clients cache file attributes and negative lookups (“this name doesn’t exist”). If a file created outside BlixtFS must appear on an NFS client immediately, mount with short attribute caching and no lookup cache. See Connecting clients.