A Long List of Distributed Filesystems

We have enough already, for crying out loud, what we don’t need is another distributed filesystems. We need to consolidate and improve on what we do have.

Here is a list of distributed filesystems that I found while looking for one that fit my needs, and some quick notes on each. (Of course I haven’t tried many of these, but please let me know if you have strong thoughts about any of them, or if you know of some glaring omissions from the list).

I’ll also say that I’m not happy with any of these yet, they all still leave plenty to be desired. If you want to talk about these please catch me at the upcoming Google Scalability Conference 08.

• Ceph

  POSIX compatible; supports data migration,re-replication when a host goes down

• coda

  con: write on close; can’t use for databases
  replaced by lustre ?

• DRBD (Distributed Replicated Block Device)

  limited to 2 servers; It is not a filesystem, but the filesystem lives on top of drbd.
  used with heartbeat(http://www.linux-ha.org/) for failover

• The GNU Cluster FileSystem (GlusterFS)

  also here
  no rebalancing or automated replication
  files are stored as whole files, not as distributed chunks there are problems with bringing an old file store online

• Google File System (GFS)

  Clearly anyone can’t just “use” this, but it deserves to be on the list because of it’s design principles, and for comparison

• gfarm

• Hadoop file system

  desgin
  win32(not production ready, but soon),linux
  written in java 1.5.x
  not designed for low latency
  write once files
  no rebalancing [yet]
  no automatic handling of namenode failure

• IBM General Parallel File System (GPFS)

• KFS

  use client library (python,…) or fuse
  comparison with Hadoop File system
  sounds promising; check back in 6 months when it’s a little more stable and full featured.

• Lustre

  posix compliant; no single point of failure
  active-active failover pairs

• memcachefs –

  Uses fuse, not persistent unless using memcachedb

• MogileFS

  write once files, replication, flat namespace, not posix compliant

• Oracle Cluster File System OCFS2

  GPL, not posix compliant, specifically designed for oracle database files

• Parallel Virtual File System (PVFS)

  requires shared storage device for redundancy; no online adding/removing of servers

• Peerfuse

  A new project. heavy development and doesn’t have enough features yet to be used in a production environment.

• Redhat Global File System –

  posix compliant, ok to use for databases

• Tahoe

  about the design
  no hotspot detection via usage
  not designed for a single user, as there is a lot of overhead in maintaining information about who as access to what files; replication is not an automatic process.

• ZFS

  not a distributed filesystem, but it has a lot of similar benefits of distributed storage devices, and it has a management interface that would be amazing for a distributed filesystem. Also see: luster

Comments are closed.

Copyright 2010 contact me