Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with OpenID
Advertise on LowEndTalk.com

In this Discussion

High-availability file system

High-availability file system

MrOwenMrOwen Member
edited October 2012 in General

I may be beating a dead [high availability] horse here, but I just wanted to put this out in the open because I don't ever recall seeing before it on LET (for a server-side solution, at least).

Was looking around a bit for replication solutions for the file system (after @netomx asked for HA solutions) and rsync is really the way to go (industry standard type stuff?) even given some of its shortcomings but I did find one sweet alternative.

I came across a project called SparkleShare. I think this is aimed at a more consumer level, but I see absolutely no reason why it can't be used to sync 2+ servers. It uses git as the backend so file syncing should happen immediately (as opposed to a rsync cron) and with less overhead because it doesn't need as much processing power as rsync.

On top of this amazingness, SparkleShare is a two-way street which means while your main server is down, the backup servers have to ability to bring the main server up to date after it comes back online.

Plus, it seems like it's under very active development.

Comments

  • I think Sparkleshare is really just a beautified version of git, which is not exactly that LEB friendly in terms of memory consumption I believe. Also, why not just use Git itself?

    http://BornIn.Asia - FREE shared hosting and subdomain service for LET members! See here for how to get one yourself!
    Have a domain that you have no use for at the moment? Click here to see how you can help! 96Forum: Low End VPS Discussions
  • @zhuanyi said: I think Sparkleshare is really just a beautified version of git, which is not exactly that LEB friendly in terms of memory consumption I believe. Also, why not just use Git itself?

    This could be true as I haven't installed it on a box yet. I'm doing so right now and I'll report back on the results when I get them.

  • zhuanyizhuanyi Member
    edited October 2012

    @MrOwen said: This could be true as I haven't installed it on a box yet. I'm doing so right now and I'll report back on the results when I get them.

    I have not installed it either but that's the feeling I got after reading their docs. But actually you made a really good point, probably the easiest way is to use one of these 2 way sync software, like Dropbox or git, since you could always launch the daemon say every 5 minutes and sync the contents. For a small website, I think Dropbox would do with 2GB of disk space.

    http://BornIn.Asia - FREE shared hosting and subdomain service for LET members! See here for how to get one yourself!
    Have a domain that you have no use for at the moment? Click here to see how you can help! 96Forum: Low End VPS Discussions
  • joepie91joepie91 Member
    edited October 2012

    Not a plug-and-play filesystem, but definitely worth a look: https://tahoe-lafs.org/trac/tahoe-lafs

    Redundant and encrypted, and decentralized to a degree.

    Appreciate my posts/software/guides? Donate (PayPal/Flattr/Bitcoin): http://cryto.net/~joepie91/donate.html | irc.freenode.net #lowendbox

    Thanked by 1MrOwen
  • MaouniqueMaounique Member
    edited October 2012

    @zhuanyi said: lunch the daemon say every 5 minutes

    No offense, just for fun :) M

    I am only representing myself :)

    Thanked by 1zhuanyi
  • @Maounique said: No offense, just for fun :)

    Sorry, it is lunch time here :) Typo corrected.

    http://BornIn.Asia - FREE shared hosting and subdomain service for LET members! See here for how to get one yourself!
    Have a domain that you have no use for at the moment? Click here to see how you can help! 96Forum: Low End VPS Discussions
  • What's the requirement?

    Is the requirement that all files are available regardless of actual uptime of the component nodes?

    Or is the requirement that no data is lost regardless of actual uptime of the component nodes?

    I am no longer affiliated with IPXcore.
  • @Damian said: What's the requirement?

    Is the requirement that all files are available regardless of actual uptime of the component nodes?

    Or is the requirement that no data is lost regardless of actual uptime of the component nodes?

    Well, preferably both but file availability is numero uno priority followed closely by maintaining next to 0 data loss. Obviously, rsync takes care of the first one but not so much the second concern.

  • fileMEDIAfileMEDIA Member
    edited October 2012

    Use Glusterfs, with 4 nodes you can build a ha raid 10 network storage or 2 servers for raid1 storage. We using it for ha vpses based on solusvm.

    fileMEDIA - German VM specialist! - https://www.filemedia.de

  • @fileMEDIA said: Use Glusterfs, with 4 nodes you can build a ha raid 10 network storage or 2 servers for raid1 storage.

    Thought about this solution already and it's pretty heavy duty and definitely *not* LEB friendly. Gluster requires a separate volume be mounted plus their recommended hardware is well above most LEB (in terms of memory mostly). Thanks for the suggestion though!

  • fileMEDIAfileMEDIA Member
    edited October 2012

    I think is a easy solution, you can mount a Glusterfs Volumen like a nfs share. You can configure two vserver for a ha storage and all should be fine. You don´t need any big configuration or something, install glusterfs from repo, create a volume and define both nodes. Mount the storage and all works. 10min or so and all should be fine.

    I use a setup with 2 vserver (50GB hdd, 1 core, 1024mb ram) and a load balancer (nginx) in front of the cluster to publish a website.

    fileMEDIA - German VM specialist! - https://www.filemedia.de

  • OCFS2 might be of interest.

    ChicagoVPS.net - OpenVZ/Xen Based - SolusVM Control Panel - TUN/PPP/FUSE/SIT/GRE - cPanel/DirectAdmin/Parallels - Chicago/Buffalo/LA Coming Soon! - Great Support!

  • You may want to take a look at OpenStack's storage solution: http://www.openstack.org/software/openstack-storage/

  • mpkossenmpkossen Administrator

    We're using MooseFS at the office and have had good experiences with it so far. It's less consumer-friendly though, but it's good, stable and has lots of options.

    Help me get free Trello Pro by signing up for an account: https://trello.com/mpkossen/recommend
    My new blog is live! Check it out at mpkossen.com

Sign In or Register to comment.