Mimiko writes:
Hello.
I would want to use a shared storage with docker to store volume so swarm containers can use the volumes from different docker nodes.
As for now I tried glusterFS and linstor+drbd.
GlusterFS has a bug with slow replication and slow volume status presentation or timing out. Also directory listing or data access is not reliable,
despite the nodes are connected with each other. So the GlusterFS didn't worked for me.
With linstor+drbd and linstor plugin for docker, the created volumes are inconsistend on creation. Did not find a solution.
DRBD might still be a solid base to run upon. Consider using an alternative file system on top of it like e.g. GFS2 or OCFS2. The management tools to
assemble these kinds of cluster file systems can be found in packages
gfs2-utils and ocfs2-utils respectively. Back when I tried it, I found OCFS2 slightly easier to setup.
I once tested a Kubernetes + OCFS2 setup (backed by a real iSCSI target rather than DRBD though) and it worked OK.
I want to try Ceth.
What other storage distributions did you tried and worked? It should have mirroring data on more than 2 nodes, representation as a bloch device to
mount in system, or as a network file system to be mounted using fstab, or using docker volume. Sould be HA and consistent with at least 3 nodes and
can automatically get back even if two of three nodes restarts, and be prune to sporadic network shortage.
I am not aware of any system that fulfils all of these requirements. Specifically, systems are typically either low-latency OR robust in event of
“sporadic network shortage” issues.
GlusterFS could be preferrable on this with there were not the bug with timeout.
I have never used GlusterFS, but from things I heard about it it very much seems as if this is not really a ”bug” but rather by design. I think that
the observed behaviour might also be similar if you were to use Ceph instead of GlusterFS, but if you find out that it works much better, I'd be
interested in learning about it :)
HTH and YMMV
Linux-Fan
öö