Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Large Disk VPS
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Large Disk VPS

Background

I am planning on converting from an HDD RAID5 array to an SDD RAID 10 array. I currently have ~5.4 TB on 3xTB drives with LVM.

I am also upgrading the mobo to handle a direct transfer of data (2 raids on the machine) due to more SATA ports.

Request

Based on this, I need a deal on a short term VPS server in which I can store my raid disk image.

I am rounding the requirement up to 6 TB to be simple. I would prefer SSD, but the cost is the priority. I have looked at DO/AWS and similar but it seems very high for temporary use.

Oh, and bandwidth would be 6-12 TB most likely. I guess an unmetered server would work, just need the transfer to not have me sitting on my ass for 2 days :P. I have fiber internet anyways.

Other

While I, of course, know I don't need to use a VPS server if I can transfer direct, I am making a backup of things before I start.

Timeline

I do not have an immediate need, as I am at the planning stage, but getting my ducks in a row, so-to-speak.

Thanks :)

«13

Comments

  • Check out hetzner auctions, I recently got an i7 (9000 passmark) 16gb ram with 6TB (2x 3TB HDD) for about 22 EUR..

    Thanked by 2uptime Hetzner_OL
  • Thanks however it does not seem to support SSH/SFTP/FTP to pipe data to it. I plan to do a dd over SSH.

  • Why not get a slice then add a slab(s)?

    VPS:
    https://buyvm.net/kvm-dedicated-server-slices/
    SLAB ($5/TB):
    https://buyvm.net/block-storage-slabs/

  • sgheghelesgheghele Member
    edited June 2019

    @pcfreak30 said:

    Thanks however it does not seem to support SSH/SFTP/FTP to pipe data to it. I plan to do a dd over SSH.

    Same product without nextcloud but with many services/protocols enabled including rsync and FTP https://www.hetzner.com/storage/storage-box

    Thanked by 1Hetzner_OL
  • uptimeuptime Member
    edited June 2019

    lunanode does hourly billing with expandable block storage - relatively affordable compared to other providers I think: $0.03 per GB per month for HDD (available in Canada and France), $0.06 per GB per month for SDD (available only in Toronto). (Remember, it's billed hourly so if you just keep the storage volume for 3 days then would work out to $0.003 per GB for those 3 days - that is, $3 per TB or $18 total for your 6 TB storage - plus whatever for the VPS and bandwidth.)

    Might check with @perennate whether they have 6 TB available in the best location for you, and how your bandwidth requirement would be handled. (Might be another $3 per TB for bandwidth - maybe you can work something out.)

    Also would check with @key900 for flexible block storage in Los Angeles (only monthly billed as far as I know)

    And (as already mentioned) @Francisco for a "slab" in Las Vegas (monthly billed, if you can get it)

    dd over SSH

    In Europe, I think renting an Hetzner auction server could be your best bet for cheap storage (monthly billed) (At the current moment, I see an i7-2600 with 2 x 3 TB HDD available for €21.85 per month - no setup fee, no extra charge for bandwidth)

    Might also check with CrownCloud (@SpeedBus) and AlphaVPS.eu (@AlexBarakov) for reliable storage options in Europe (possibly also Los Angeles for CrownCloud)

    EDIT2:

    In the US, another option might be with @HostDoc in Dallas. It could be possible to arrange for something on a 10 Gbit pipe. Which is nice.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    Slabs are love.
    Slabs are life.
    Slabs are sold out (until next week).

    Francisco

  • @Francisco said:
    Slabs are love.
    Slabs are life.
    Slabs are sold out (until next week).

    Francisco

    Thanks. I think I'm preferring to pay 30-40 for 1 month vs risking this taking a week longer than I expect and costing 5x as much on per hour rates. It's not urgent anyways so can wait on new stock :P.

  • @pcfreak30 said:

    @Francisco said:
    Slabs are love.
    Slabs are life.
    Slabs are sold out (until next week).

    Francisco

    Thanks. I think I'm preferring to pay 30-40 for 1 month vs risking this taking a week longer than I expect and costing 5x as much on per hour rates. It's not urgent anyways so can wait on new stock :P.

    https://buyvmstock.com/

    Thanked by 1uptime
  • uptimeuptime Member
    edited June 2019

    (was a blank post due to hitting wrong button, sorry - nevermind, nothing to see here)

  • @webclouddev said:

    @pcfreak30 said:

    @Francisco said:
    Slabs are love.
    Slabs are life.
    Slabs are sold out (until next week).

    Francisco

    Thanks. I think I'm preferring to pay 30-40 for 1 month vs risking this taking a week longer than I expect and costing 5x as much on per hour rates. It's not urgent anyways so can wait on new stock :P.

    https://buyvmstock.com/

    Thanks, that covers slices but not "slabs" :P. already got a $2 server just now.

  • @pcfreak30 said:

    @webclouddev said:

    @pcfreak30 said:

    @Francisco said:
    Slabs are love.
    Slabs are life.
    Slabs are sold out (until next week).

    Francisco

    Thanks. I think I'm preferring to pay 30-40 for 1 month vs risking this taking a week longer than I expect and costing 5x as much on per hour rates. It's not urgent anyways so can wait on new stock :P.

    https://buyvmstock.com/

    Thanks, that covers slices but not "slabs" :P. already got a $2 server just now.

    that will be easy i guess once they restock slab since you already have slice

  • pcfreak30 said: I need a deal on a short term VPS server in which I can store my raid disk image.

    Instead of Hetzner nextcloud (suggested earlier) why not Hetzner storage? 5 TB (nothing for you to do - except sftp the image in) is ~ €21.90 for a month and gives you proper redundant storage and enough bandwidth. Of course you can go the dedi route but then you've to setup the machine and you're sort of relying on the disk to stay alive (beware of Mr. Murphy).

    Thanked by 1uptime
  • uptimeuptime Member
    edited June 2019

    @nullnothere - I believe OP wants to dd via ssh which probably wouldn't work as such directly on a hetzner storagebox

    EDIT2: with all due respect to Mr. Murphy, indeed ...

  • fail..

    @nullnothere I would need the 10 TB option as my disk array is MORE than 5 TB. please read OP again :P.

  • @uptime yes thanks. I am having to get a new mobo as is for this setup and I don't need to get a 15 SATA mobo which I'm not sure even exists + 500 in new HDD's for backup one time :D

    So I need to send it up streaming to a remote box since I can't store the image locally at all.

    Hope all is clear now :)

    Thanked by 1uptime
  • uptimeuptime Member
    edited June 2019

    @pcfreak30 - just a thought - might at some point consider investing in an usb3 external 8 TB "easystore" drive for (on-site) backups (see https://www.reddit.com/r/DataHoarder/comments/7fx0i0/wd_easystore_8tb_compendium/)

    ah ... looks like it's currently priced around $170 - I got one shipped (in the US, maybe a year and a half ago) for about $152 sale price including tax.

    I know it's not exactly a proper "backup" if it's still on-site but it's one leg of that journey

  • @uptime said:
    @pcfreak30 - just a thought - might at some point consider investing in an usb3 external 8 TB "easystore" drive for (on-site) backups (see https://www.reddit.com/r/DataHoarder/comments/7fx0i0/wd_easystore_8tb_compendium/)

    ah ... looks like it's currently priced around $170 - I got one shipped (in the US, maybe a year and a half ago) for about $152 sale price including tax.

    I know it's not exactly a proper "backup" if it's still on-site but it's one leg of that journey

    Thanks though I have a feeling a usb would be a bigger bottleneck than dd -> fiber. Plus When im done I will have 3 unused HDD's.

    So I think thats a bit of a waste as I wont be using the vps for more than a month hopefully.

    Thanked by 1uptime
  • exception0x876exception0x876 Member, Host Rep, LIR

    If you are ok with a temporary custom solution, please feel free to open a support ticket at wishosting.com

    Basically, I can increase the storage space of KVM Storage CA package to 6TB+ since there is some free space on the nodes.

    Thanked by 1dahartigan
  • @exception0x876 what would the price be?

  • exception0x876exception0x876 Member, Host Rep, LIR

    @pcfreak30 said:
    @exception0x876 what would the price be?

    I can create a quote if you specify the required storage and the duration of the lease.

    Thanked by 1uptime
  • letboxletbox Member, Patron Provider

    Budget?

  • Given the best offer I have seen is @Francisco $5/TB, looking at 6 TB. So ~$35 or less, and likely under 2 weeks, definitely shorter if nothing fucks up. Just needs to hold up til my data migration is done :D.

    @exception0x876 @key900

  • letboxletbox Member, Patron Provider

    @pcfreak30 said:
    Given the best offer I have seen is @Francisco $5/TB, looking at 6 TB. So ~$35 or less, and likely under 2 weeks, definitely shorter if nothing fucks up. Just needs to hold up til my data migration is done :D.

    @exception0x876 @key900

    We already have the same price rate $5/TB! We have stock if you are interested.

  • letboxletbox Member, Patron Provider

    @uptime said:
    out.)

    Also would check with @key900 for flexible block storage in Los Angeles (only monthly billed as far as I know)

    >

    Thanks for recommendation:)

    Thanked by 2uptime skorous
  • pcfreak30pcfreak30 Member
    edited June 2019

    @key900 Thanks, depending on availability will use you or @Francisco likely when its time

  • @uptime said: I believe OP wants to dd via ssh

    @pcfreak30 said: I would need the 10 TB option as my disk array is MORE than 5 TB.

    Aah - my bad. I didn't really read the "fine" print.

    I'm still kind of thinking you're better off with rsync, maybe even borg to a remote location (certainly not a no-raid dedi) rather than a raw disk image. It gives you much more flexibility in chunking your backups, staging it, recovering in parts (like getting the OS up and running right away and then maybe restoring the data over a couple of days or whatever). This will also give you much better resilience in case something bad happens in between as you have various "parts" of your backup live without needing to do any image trickery. My $0.02 - at least I would be very uncomfortable with a mega monolithic and unwieldy backup which I feel is very limiting and inflexible.

    Plus if you go with @Francisco such that you order say mid month (assuming you get stock of course!) , you'll anyway only be billed pro-rated for the rest of the month - so you get all the flexibility+reliability you want without really loosing anything and yet being able to bring up your setup incrementally and hopefully painlessly.

  • @nullnothere said:

    @uptime said: I believe OP wants to dd via ssh

    @pcfreak30 said: I would need the 10 TB option as my disk array is MORE than 5 TB.

    Aah - my bad. I didn't really read the "fine" print.

    I'm still kind of thinking you're better off with rsync, maybe even borg to a remote location (certainly not a no-raid dedi) rather than a raw disk image. It gives you much more flexibility in chunking your backups, staging it, recovering in parts (like getting the OS up and running right away and then maybe restoring the data over a couple of days or whatever). This will also give you much better resilience in case something bad happens in between as you have various "parts" of your backup live without needing to do any image trickery. My $0.02 - at least I would be very uncomfortable with a mega monolithic and unwieldy backup which I feel is very limiting and inflexible.

    Plus if you go with @Francisco such that you order say mid month (assuming you get stock of course!) , you'll anyway only be billed pro-rated for the rest of the month - so you get all the flexibility+reliability you want without really loosing anything and yet being able to bring up your setup incrementally and hopefully painlessly.

    My boot and os are on SSD m.2 already, I avoided THAT trap. Only the home is on RAID. It is LVM using a raid physical disk so-to-speak. So I plan on dd'ing an LVM snapshot and stay working as numbers show it will be a long day of data copy. Then it's just playing musical chairs with the lvm group and transferring extents.

  • @pcfreak30 said: Only the home is on RAID. It is LVM using a raid physical disk so-to-speak. So I plan on dd'ing an LVM snapshot and stay working as numbers show it will be a long day of data copy.

    I think you'll still be better off with vanilla rsync.

    Back of the envelope calculations:

    a) 5 TB = 5 x 1024 = 5120 GB = 5242880 MB
    b) Assuming 100MB/s transfer (sustained - tune down if too much), time to copy is ~15 hours. If you go down to 60MB/s, you're at ~24hours. This is assuming local-disk-reads+network-out+remote-disk-write. 60MB/s sustained network writes are not that trivial to expect - so I suspect (depending on where you are latency etc.) you'll be at possibly 30-35 MB/s average overall (of course if you're lucky it'll be higher!).
    c) With rsync, you can still do this in pieces and with control/recovery (as well as incremental painless updates in case of local modifications or network blips as you're doing the remote backup)
    d) Assuming all rsyncs are finished, replace local HDDs with SSDs, repeat backwards again in pieces (after due LVM setup).

    My reason to prefer rsync is if there's anything bad on the network side, I don't loose anything and can continue from where I left off.

    So you're definitely looking at maybe 2 days to write data out and then another 2 to bring it back in - a good 1 week project overall perhaps, but not really needing your attention all the time of course.

    Good luck and may Mr. Murphy not be with you.

    Thanked by 1uptime
Sign In or Register to comment.