Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with OpenID
Advertise on LowEndTalk.com

In this Discussion

backup server-> What kind of config?

backup server-> What kind of config?

TazTaz Disabled
edited July 2012 in General

Planning to build 2 backup servers. Planning to use below configuration, 8x2tb sata Intel i3 6gb ram 10g port dedicated.

What would be your suggestion regarding raid (1, 10 or 5)? Since this will only be used for backup purpose do I need more ram? What would be the best method? Scp, FTP or rsync ( assuming server will be accessed on a 2 hour basis). do I need more ram? Anything else I need to worry about? Anything else you want to suggest?

Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

Thanked by 1inverse

Comments

  • KuJoeKuJoe Member
    edited July 2012

    6GB is more than enough (our backup server has 256MB of RAM and 1 CPU core). RAID5 would be good, RAID6 would be better. rsync is best to conserve bandwidth/CPU.

    My only recommendation is to make your backup server an OpenVZ container (even if it's the only container on the server and has 100% of the resources assigned to it). This will make it easier to move if you need to (i.e. hardware failure, upgrades, etc...). Making it a container will make sure that the backup OS will not be dependent on hardware or configs so if say you need to rebuild the RAID from scratch (i.e. add more hard drives), you can just vzmigrate to another OpenVZ host with enough disk space, rebuild the array from scratch, and vzmigrate it back without data loss.

  • TazTaz Disabled

    @kujoe great advice! I will surely do that.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • Just a question - is it for backups of your clients or you intend to sell backup containers?

  • TazTaz Disabled

    @Alex_liquidhost this is for clients backup only. Either on a node level or per clients based FTP storage for free. Still not decided. But no, no backup container.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • Oh, thats quite alot of storage. I do backups every 24 hours and can fit all our data in around 2.5TB :P

  • TazTaz Disabled

    @Alex_liquidhost, considering the fact that we have couple of fully loaded nodes with around 1.5tb usage, these two backup servers are going to do my task for a coupe of solid months before we decide to expand.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • krokro Member

    Kudos on a normal thread ninja

    Thanked by 2TheHackBox eastonch
  • TazTaz Disabled

    @Kro sorry to disappoint. :P

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • rds100rds100 Member

    If you don't need all that space and can live with about 8TB of storage for your backups i would go for several RAID1 volumes. RAID1 gives you the ability to relatively easy recover the data even in case of major hardware failure (i.e. if the RAID card dies, etc.) - you can just plug the individual disks from a RAID1 volume in another servers and recover the data.

    Thanked by 1Taz
  • @rds100 said: If you don't need all that space and can live with about 8TB of storage for your backups i would go for several RAID1 volumes. RAID1 gives you the ability to relatively easy recover the data even in case of major hardware failure (i.e. if the RAID card dies, etc.) - you can just plug the individual disks from a RAID1 volume in another servers and recover the data.

    This. And make those all containers. Sure, if it is for customers. If it is for dc backup, then a larger raid will do better than a few smaller ones. A scheme can be implemented to span the backup on more containers. If we worry for card failure, a raidz-raidz2 will probably put everything at ease. M

    I am only representing myself :)

    Thanked by 1Taz
  • camargcamarg Member
    edited July 2012

    You're ok as far as ram and cpu is concerned raid6 would be my choice about the software to use take a look at rsnapshot, it's great

    have a look at the thread I opened when I was setting up my backup server http://www.lowendtalk.com/discussion/924/software-raid

    Thanked by 1Taz
  • I think I would personally use rSync on that server, and as suggested, within a container, makes it easy to modify etc, you can assign it "certain" amounts of space and then you can clearly see when you're going to need more, it also saves the need to get something to monitor bandwidth. (I would presume you're throwing SolusVM onto this for ease?).

    Everything is fine, your ram might be a little OTT, but that's fine. Disk space is fine and personally would also reccomend Raid 6.

    Question though, would this server be located at the same location as the server it's backing up? Or would it be on a remote location?

    If it's the latter, then your 10G port is still rather extreme, unless you are backing up 3-5 server that are on 1Gbit ports.

    If it's the former, then why do you need such a large connection, still? You could just use internal IP's and internal networks and not have to actually pay for bandwidth usage, if your DC allows internal traffic like that. (I know from experience, Secured Servers allowed me to do this).

    Good luck, and it's great to see a relevant thread from you!

    Thanked by 1Taz
  • TazTaz Disabled

    @estonch Backups will be done using cron and good old manual way. We will most likely provide free ftp space instead of doing it ourself but, still deciding what to go for.

    Backup servers will be on a different DC. We decided to keep them separated so if for somereason, main dc falls under the wrath of muphey, we can still get those backups and get going with in no time. Atleast 4 servers will be connected to these box at any given time. So 10g would be perfect for our need.

    @camarg Thanks.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • pcanpcan Member

    I strongly suggest to use CPU, motherboard and RAM with ECC support. A bad memory stick could spell disasters on a backup server. Your bottleneck will be the disk transfer rate, and will be way lower than 10 gbps if you choose slow 2TB drives. For a comparison with real world applications: with 12 x SAS drives at 15krpm in RAID10 I am unable to saturate 4 balanced (Round Robin) gigabit connections. Raid 6 will have a worse write speed than Raid 5, I suggest you to test both configurations, and then make a choice. A hot spare disk is a life saver. I fully endorse the KuJoe suggestion to use a container.

    Thanked by 1Taz
  • TazTaz Disabled

    @PCAN good point regarding HD. SAS indeed would be a better solution. Time to increase my budget.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • pcanpcan Member

    To increase throughput, you can choose SAS (it will also decrease access time, not really of great concern on backup servers) or increase the number of drives and SAS/Sata links.

    You can get some ideas from the well-known backblaze design, see http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/

Sign In or Register to comment.