Key concepts
The handful of ideas you need to read the rest of the documentation.
Bucket
A container of objects in a cloud object store. BlixtFS serves each configured bucket as a directory in the filesystem. Azure calls a bucket a container.
You name a bucket with a URL-like bucket spec. The scheme picks the provider and the attributes after the colon change how it is served:
s3://my-bucket
gs://training-data:readonly,lazy
r2://media-archive:nopubsub
Buckets lists every scheme and attribute.
Object and file
Each file in BlixtFS is exactly one object in the bucket. The object key is the
file’s path relative to the bucket. Directories are stored as zero-byte
directory marker objects whose key ends in /. You can therefore read the
bucket without BlixtFS, and objects written by other tools show up as files.
Metadata index
Blixt builds and maintains an in-memory index that is also persisted to disk. The index records every file and directory. The index stores file attributes, such as: name, size and permissions. Key attributes are stored as object metadata. The bucket is always the source of truth: The metadata index can always be rebuilt from the bucket.
Indexing and lazy indexing
When BlixtFS first serves a bucket it lists the bucket into the index. Large buckets
can use lazy indexing. BlixtFS then serves the bucket immediately and lists each
directory the first time someone opens it, while a background crawler fills in the rest.
Change notifications
Most object stores can publish an event whenever an object is created or deleted. BlixtFS subscribes to these events, so changes made directly in the bucket appear in the filesystem, usually within seconds. Providers that offer no notifications are kept in step by periodic rescans. See Change notifications.
Consistency check (fsck)
A background job that compares the index with the bucket and reconciles any difference. It runs periodically. Additional consistency checks can be requested for a directory. See Consistency checks.
Protocol gateway
The component a client actually connects to: an NFS server, an SMB server, etc. Each gateway translates its protocol into calls to the file servers.
Single node and scale-out
A single-node deployment runs all BlixtFS services in a single container. It is the simplest way to run BlixtFS and is enough for many teams. A scale-out deployment runs a collection of services. Each service can grow or shrink independently without downtime. Capacity in terms of request rates or total size scales independently for metadata, data reads, data writes, etc. See Scale-out.
Edition and license
BlixtFS comes in Standard (free), Enterprise and High Performance editions. A license file unlocks the paid features. Without a valid license, BlixtFS runs as the Standard edition. See Editions.