Their architecture descriptions starts with a strawman:
> Usually distributed file systems split each file into chunks, a central master keeps a mapping of filenames, chunk indices to chunk handles, and also which chunks each chunk server has.
> The main drawback is that the central master can't handle many small files efficiently, and since all read requests need to go through the chunk master, so it might not scale well for many concurrent users.
The chunk server architecture has been first put into production with the Google File System AFAIK. And it has been designed specifically for large files (what search needed at that time). So no surprise.
But that's only one architecture for a DFS. There are also block-based DFS (like GPFS),
object-based DFS (Lustre.), cluster file systems (OCFS), and other architectures. They exhibit other characteristics.
Telling from the architecture and Wiki, it does not seem to be a file system at its core, but an object store with a file translation layer. One of the core problems of this approach is that in-place updates usually mean read-modify-write (if the object store has immutable objects, like most have, with Ceph being a notable exception).
From the replication page:
> If one replica is missing, there are no automatic repair right away. This is to prevent over replication due to transient volume sever failures or disconnections. In stead, the volume will just become readonly. For any new writes, just assign a different file id to a different volume.
This sounds like the architecture and implementation is still pretty basic. Distributed storage without redundancy (working redundancy!) is not that interesting.
Sorry to be that critical (great that someone writes a distributed file system!), but I think it is important to add some context. And the seaweed auther seems to have problems with bold statements either...
Disclaimer: I also work on a distributed file system (with unified access via S3 ;)
There are actually no limit for blob size. The clients decide the blob size. It is limited by available memory and concurrency.
By default, the filer client uses 8MB.
Each identifier is as <volume_id, file_key, cookie>. With volume_id, you can locate the volume server. So the volumes are portable and can be moved around.
For a distributed filesystem's redundancy to be working in my book, it has to actually be able to reconstruct itself after unavailability/failure, which Seaweed seemingly doesn't do.
Additionally, something more complex than N-replication (eg. M.N erasure coding) is also a strong 'redundancy working' requirement for me.
a) I wonder in which sense HDFS was ever the most popular DFS. You could argue that it is not even a DFS, because it is not a general purpose / POSIX file system. In 2012 there was Lustre, GlusterFS, GPFS, ... Not many DFS were built since 2012.
b) It is completely unclear how the replication works, and which properties it has (split-brain safe?). From what it states (see citation), the "repair" is not automatic.
a) you are right. I was not in storage field.
b) During writes, the strong consistency is required. The repair is by admin scripts, not by the data node themselves. This is to keep data nodes simple and scalable. You can also run the repair scripts at the "safe" time.
> Usually distributed file systems split each file into chunks, a central master keeps a mapping of filenames, chunk indices to chunk handles, and also which chunks each chunk server has.
> The main drawback is that the central master can't handle many small files efficiently, and since all read requests need to go through the chunk master, so it might not scale well for many concurrent users.
The chunk server architecture has been first put into production with the Google File System AFAIK. And it has been designed specifically for large files (what search needed at that time). So no surprise.
But that's only one architecture for a DFS. There are also block-based DFS (like GPFS), object-based DFS (Lustre.), cluster file systems (OCFS), and other architectures. They exhibit other characteristics.
Telling from the architecture and Wiki, it does not seem to be a file system at its core, but an object store with a file translation layer. One of the core problems of this approach is that in-place updates usually mean read-modify-write (if the object store has immutable objects, like most have, with Ceph being a notable exception).
From the replication page:
> If one replica is missing, there are no automatic repair right away. This is to prevent over replication due to transient volume sever failures or disconnections. In stead, the volume will just become readonly. For any new writes, just assign a different file id to a different volume.
This sounds like the architecture and implementation is still pretty basic. Distributed storage without redundancy (working redundancy!) is not that interesting.
Sorry to be that critical (great that someone writes a distributed file system!), but I think it is important to add some context. And the seaweed auther seems to have problems with bold statements either...
Disclaimer: I also work on a distributed file system (with unified access via S3 ;)