Buckets
Name the buckets BlixtFS serves, and control how each one is indexed, updated and replicated.
Naming buckets
BlixtFS serves the buckets you name when it starts. Name them as positional
arguments, in the BUCKETS environment variable, or in the configuration file.
Naming a cloud without naming any of its buckets — an account or project plus credentials — serves every bucket those credentials can see.
Naming nothing at all is not an error. BlixtFS starts and serves a single file in the mount root:
/no_buckets_specified.txt
The file explains how to name buckets and links back to this page. It is read-only and disappears as soon as BlixtFS is restarted with at least one bucket, so an empty mount always explains why it is empty.
Bucket specs
On the command line and in environment variables, a bucket is named with a bucket spec: a scheme, the bucket name, and optional attributes after a colon.
scheme://bucket-name[:attribute,attribute=value,...]
docker run ... blixtfs/standard:2.11.0 \
s3://my-bucket \
gs://training-data:readonly,lazy \
r2://media-archive:rescan=6h
Schemes
| Scheme | Provider | Directory under the mount root |
|---|---|---|
s3:// |
Amazon S3 | aws/ |
gs:// |
Google Cloud Storage | gcp/ |
azure:// |
Azure Blob Storage | azure/ |
oci:// |
Oracle OCI Object Storage | oracle/ |
minio:// |
MinIO | minio/ |
r2:// |
Cloudflare R2 | cloudflare/ |
cw:// |
CoreWeave AI Object Storage | coreweave/ |
A bucket name without a scheme belongs to the cloud named by --default_cloud.
The directory root names can be changed.
Attributes
| Attribute | Meaning | Available |
|---|---|---|
readonly |
Never write to this bucket. See Read-only buckets. | 2.8 or earlier |
ro |
Short for readonly. |
2.9 |
anonymous |
Read without credentials, for public datasets. Implies readonly. |
2.9 |
lazy |
Serve immediately; index each directory on first access. | 2.8 or earlier |
pubsub, nopubsub |
Turn change notifications on or off for this bucket. | 2.8 or earlier |
topic=NAME |
Subscribe to an existing notification topic instead of creating one (GCS and S3). | 2.9 |
rescan=DURATION |
Relist the bucket on a schedule, for example rescan=1h. rescan=never turns it off. |
2.9 |
read_replication=N |
Number of cache copies of each chunk (scale-out). | 2.8 or earlier |
write_replication=N |
Number of write-tier copies of unuploaded data (scale-out). | 2.8 or earlier |
uid=N, gid=N |
Default owner and group of files in this bucket. See Default owner and permissions. | 2.11 |
file_mode=OCTAL, dir_mode=OCTAL |
Default permissions of files and directories, for example file_mode=0640. |
2.11 |
The readonly attribute has existed for several releases, but the guarantee
that BlixtFS never writes to the bucket in any way is complete from 2.9. See
Read-only buckets.
If a bucket spec contains an attribute BlixtFS doesn’t recognise, that bucket is not served, and the log explains why. The other buckets are served as usual. A typo can never leave a bucket served writable or unindexed: the bucket is left out entirely rather than served with the attribute ignored. (Releases before 2.9 ignored the attribute, and 2.9.0 refused to start.)
BlixtFS serves whole buckets. A spec that names a path inside a bucket, such
as r2://media-archive/2024, is skipped the same way, and the log names the
bucket to configure instead. This applies in the configuration file and
environment variables too.
Buckets in the configuration file
In YAML, each bucket is an entry under its provider, and attributes become keys:
cloud:
aws:
enabled: true
buckets:
- bucket: s3://my-bucket
region: us-east-1
- bucket: s3://shared-data
region: eu-west-1
read_only: true
rescan: 1h
topic: arn:aws:sns:eu-west-1:123456789012:owner-events
gcp:
enabled: true
project: my-project
buckets:
- bucket: gs://training-data
lazy: true
- bucket: gs://public-dataset
anonymous: true
- bucket: gs://team-share
uid: 1000
gid: 1000
file_mode: 0660
dir_mode: 0770
S3 bucket entries need the bucket’s actual region. To serve every bucket the
credentials can see, set all_buckets: true on the provider instead of listing
them.
Default owner and permissions
Each bucket has a default owner, group, file mode and directory mode.
A new bucket takes its defaults from the server-wide filesystem settings:
filesystem:
default_uid: 1000
default_gid: 1000
default_file_mode: 0644
default_directory_mode: 0755
The uid, gid, file_mode and dir_mode attributes override them for one
bucket. Unlike the server-wide settings, they also apply to a bucket that
already exists: change one, and every file that had the old default changes
with it. Files with any other owner or mode are not affected.
Removing an attribute leaves the bucket’s current default in place.
The files that change get a new change time (ctime), as they would with
chown or chmod. BlixtFS serves the new owner and modes within a second or
two of updating them. Mounted clients cache attributes, though, and can show
the old values until their cache expires. A Linux NFS client keeps them for up
to 60 seconds by default. Mount with a smaller actimeo to see changes sooner.
Modes are always octal, so 644 and 0644 are the same. A bad value means the
bucket is not served, as with other attributes.
Environment variables
Buckets can also be listed in BUCKETS, or per provider in AWS_BUCKETS,
GCP_BUCKETS, AZURE_BUCKETS, ORACLE_BUCKETS, MINIO_BUCKETS,
CLOUDFLARE_BUCKETS and COREWEAVE_BUCKETS. Separate several buckets with
spaces or semicolons. Attributes work the same way as on the command line:
BUCKETS="gs://training-data:readonly,lazy gs://team-share:uid=1000,file_mode=0660"
A comma separates a bucket’s attributes, never buckets. A spec with a comma
in its bucket name, such as gs://a,gs://b, is not served, and the log
explains why.
Provider directories
The “global” mount root has a directory for every cloud provider,
as described in “Directory under the mount root” above.
By default, only configured providers are listed.
Any other provider’s directory is there too, and works by name.
Once a client names it – looks it up (stat, cd, a path through it)
or makes it with mkdir – the “global” mount root lists it.
ls /mnt/blixt/oracle
It stays listed, on every server and across restarts.
A mkdir of a name that isn’t a provider is refused.
A provider you have enabled is always listed. Showing a provider doesn’t enable it: its buckets still need credentials. See Cloud credentials.
SMB clients may not list a provider that another client named until they reconnect: they keep serving the “global” root’s listing they already have.
Creating buckets from the filesystem
A mkdir directly under a cloud’s directory in the “global” root means make this
bucket available:
mkdir /mnt/blixt/minio/new-bucket
- If the bucket already exists — it is loaded: probed and served exactly as it is when listed, read-only included.
- If the bucket does not exist — it is created, if supported by the object store API. Some cloud providers (AWS, GCP, Cloudflare and CoreWeave) have no creation path at all: their buckets are made in the provider’s console or with its CLI. See LINK for details.
Serving buckets from the filesystem
Accessing a bucket under a cloud’s directory in the global root makes it available, even if the bucket wasn’t explicitly configured.
$ ls /mnt/blixt/aws/
configured-bucket
$ ls /mnt/blixt/aws/other-bucket/
data/ index.json
$ ls /mnt/blixt/aws/
configured-bucket other-bucket
BlixtFS probes the bucket first: it has to exist and its contents have to be
listable. A bucket that fails either is ENOENT, with the reason in the server
log.
There is one refusal that is not ENOENT. If the cloud will not say whether the
bucket exists – S3 answers 403 for a bucket in an account your credentials
cannot see, deliberately, so that a bucket name cannot be tested for – then
BlixtFS does not know whether it is there, and says so: the listing fails with
EPERM (EACCES over FUSE) rather than claiming the name is free. A mkdir of
that name fails the same way instead of trying to create a bucket whose name is
taken. Credentials the cloud rejects outright are reported the same way, which is
the case worth watching for in the log: it is a configuration problem, and every
bucket on that cloud is about to fail with it. It is then served in lazy mode, so the listing
that found it is answered immediately while the rest of the namespace fills in
behind it.
A bucket BlixtFS cannot write is served read-only, which is how a public
dataset is mounted: nothing is written to it, not even the directory markers and
metadata stamps a writable bucket gets, and every write from a client is refused
with EROFS. The decision is stored with the bucket, so it holds after a
restart and on every other server — the same guarantee a bucket configured with
:ro gets.
The bucket now has an entry in the index, and every bucket in the metadata index is served.
Nothing marks it as special: it is indexed, swept and watched exactly like
a bucket named in the configuration, and so is a bucket created with mkdir.
In a scale-out deployments new bucket discovery is managed centrally. New buckets are made available to all servers at once.
Things worth knowing:
- Disable bucket discovery by setting
indexing.bucket_discoverytofalse. - Any client with a mount can discover buckets. Discovery costs cloud API calls and metadata rows, and the write probe puts one small object into a bucket nobody configured — under BlixtFS’s internal prefix, and deleted again.
- Live change notifications need write access, so a bucket served read-only this way has none: configuring them is itself a write. Its changes are picked up by rescans instead.
- A public bucket the configured credentials cannot see at all is still
ENOENT. Discovery probes with the credentials the server already has; reading a bucket with no credentials at all is theanonymousattribute, and that has to be configured. - To stop serving a bucket, remove it. See Removing buckets.
This is the per-name counterpart of all_buckets, which enumerates every
bucket the credentials can list at startup.
Removing buckets from the filesystem
Setting the extended attribute bfs.remove_bucket on a bucket’s directory
stops serving the bucket. Available from 2.11.
setfattr -n user.bfs.remove_bucket -v 1 /mnt/blixt/aws/other-bucket
The bucket’s entry in the index is deleted and it leaves the provider directory’s listing, on every server. The bucket itself is never touched: its objects stay where they are. Looking the bucket up again serves it again and indexes it from scratch, like any other discovered bucket. The value is ignored.
- Only a bucket’s own directory. The attribute is refused with
EINVALon anything else, such as a file or directory inside a bucket, a provider directory or the mount root. It is refused on its own too, if the same call sets other attributes. - Not while the bucket is in use. While a file in the bucket is open, a
write is still uploading, or a consistency check of the bucket is running,
the attribute is refused with
EINVAL, and the server log says why. Close the files, wait for the uploads to finish, and try again. - A read-only bucket can be removed too. Removing writes only to the index, never to the bucket.
- A configured bucket comes back. A bucket named on the command line, in
an environment variable or in the configuration file is served again when the
server that reads that configuration restarts, and so is every bucket of a
provider with
all_buckets. To remove one for good, take it out of the configuration too. - Change notifications stop, but what was set up in the provider for them stays, and is used again if the bucket comes back.
- Any client with a mount can remove a bucket, for every other client.
The file protocols do not say who is asking. Turn the attribute off by
setting
indexing.bucket_removaltofalse. In a scale-out deployment the config server’s setting decides.
Which clients can set the attribute:
- Linux clients over NFSv4 and SMB, and FUSE mounts. NFSv3, SFTP and WebDAV
have no extended attributes, and macOS has no
user.namespace. - FUSE takes the name without the
user.prefix:setfattr -n bfs.remove_bucket -v 1 /mnt/blixt/aws/other-bucket.
To delete an empty bucket from the provider as well, rmdir its directory
instead, where the provider allows it.
Indexing large buckets
The first time BlixtFS serves a bucket, it lists every object into the index before serving it. For buckets with millions of objects that can take a long time. Two ways to start faster:
- Mark the bucket
lazy. It is served immediately. Each directory is listed the first time someone opens it, and a background crawler indexes the rest. - Set
indexing.lazy: trueto make every bucket lazy.
Some cloud providers (GCP) provide API calls to list objects faster with metadata attributes. See LINK for details.
Keeping up with outside changes
By default BlixtFS subscribes to the bucket’s change notifications (see
Change notifications). For buckets that
can’t deliver them, or where you prefer not to, use nopubsub together with
rescan=. The periodic consistency check
also catches anything missed.