cetrafs
ExplanationUnderstanding-oriented

Concepts

Why cetrafs works the way it does: the design, the trade-offs, and how it compares to VMFS and GFS2.

These pages explain the design. Read them away from the keyboard, when you want to understand why cetrafs behaves the way it does, what it promises, and where its guarantees stop. For exact values, see Reference.

A suggested reading order: start with the architecture overview, then cluster coordination and the locking model. The rest can be read in any order.

Architecture overview The shape of cetrafs: independent nodes, one shared disk, no network between them, and an on-disk layout built for compare-and-write. Cluster coordination How nodes coordinate with nothing but the shared disk: atomic compare-and-write, the Node Registry, heartbeats and the stale window. Zone Groups and allocation Why each node allocates only from regions it owns, how ownership moves, and how a block in someone else's region gets freed. Locking model Whole-file exclusive locks instead of byte-range locks, per-inode namespace locks, read presence, nofilelock for live migration, and flock. Journaling and crash consistency The per-node write-ahead redo journal, why replay is safe on a peer's behalf, the replay guard, and what 'per-fsync, not per-syscall' means. Fencing and split-brain Why a heartbeat timeout can't tell a dead node from a slow one, what cetrafs self-fencing protects against, and why hung hosts must be powered off. Checksums and integrity CRC32C on every on-disk structure, verified by the kernel on read, the torn-read retry, inode generations, and how fsck decides what it may repair. Performance model Why data throughput is high and metadata operations are slow, what the measurements show, and how to benchmark cetrafs honestly. cetrafs vs VMFS Where cetrafs matches VMFS's clustering model, where it goes further, and where it deliberately does less. cetrafs vs GFS2 Two different answers to two different questions: when GFS2 is the right choice, when cetrafs is, and measured throughput on the same LUN. Proxmox without Ceph The options for shared Proxmox VE storage on an existing iSCSI or Fibre Channel SAN, without Ceph: shared LVM, NFS, ZFS over iSCSI, GFS2/OCFS2 and cetrafs.