New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
$3k/month for 600tb of servers and disk space sounds doable. 10x Hetzner SX290 servers (15x 6TB drives with hardware raid controllers) would run you 2500 euro with 900 (decimal) TB of raw disk, letting you put up a Ceph or comparable cluster with underlying raid-6, if my calculations are right. There is enough included bandwidth too.
Are you able to sign a 12-month contract? Do you need any Raid on these drives or JBOD?
--
It is not, no, but it limits at high IO which here seems a given; 3Gbps on 10G traffic out needs at least 10GE also for the storage network.
Have seen it at 20Gbit Infiniband (this colo), 1GE, 10GE and 40GE with Infiniband coming out far above 10GE bonded *2 and less CPU heavy than 40GE (which has higher speed which is useful even just with a few SSDs or 20+ HDDs).
You have... certain issues with this setup but yes overall the calculation is right and ceph works 'fine'; we have something similar with smaller Hetzner servers (eg. 4x4TB) and in colo similar at one point. The SX291 if you disregard (or calculate in) the setup is one of the cheapest storage options available.
Issues arise of the 1Gbit internal network on this large nodes and on the traffic outbound being split evenly (Hetzner has no pooling) which A) limits to either files on one IP to one server BW or
uses internal network to balance traffic usage across the backends - which is the same 1G port as external and thus limits you external BW available.
It also does not let you spike over 1Gbit for a file stored on node A unless you cache on B/C in any config (just think about it, the 1G port and serving direct/from other server).
So smaller servers with more overall BW (as Hetzner does not really provide 10G ports, or not at realistic prices where available) make more sense.
That was a very dirty hit below the belt.
Doing cheap cheap cheap storage for a gazillion customers on a gazillion vps (and ovz at that) - is very different from - doing a properly funded and profitable medium to high end customer project.
You know whats also dirty? Having customers offline for a month and ignoring their tickets.
My brands have had more than their share of issues ranging from failing hardware to incompetence on my end, but every time, I at the very least keep customers well updated on the progress of things and what we're doing to address the issue.
At 1Gbit/sec sustained you can move around 8TB+ a day from these nodes to one of the other 10's of nodes he claimed he provisioned. The people with issues are on nodes that are online, just, they're missing half or more of what they paid for (look into any of the shrinkage tickets). This isn't even counting vel7 which will never finish FSCK'ing.
Ultimately, in the end the only reason anyone would do storage plans on OpenVZ is because they either have a large swatch of space spare (so it's a limited thing), or they're overselling the shit out of things. It's a nightmare when (not if, when) you have to FSCK the volume or you end up with multi page threads discussing the ongoing availability issues.
I'm rooting for him to address it and prove his worth, but we've seen him try to post SSD sales threads and now try to get this customer (one which is entirely based around huge storage) all the while his users are not getting service.
Francisco
You just sliced him
I more or less agree with William but I come to another conclusion.
One very important factor is the scale of operations. If one, like e.g. cociu, anyway has considerable storage products (like storage vps) in operation, the equation changes in some considerable points.
If the customer jumps off (or is taken out by lea) the hardware investment isn't much of a burden and can be partitioned differently and put into other products (e.g. e.g. storage vps or a centralized backup offer, etc).
Moreover, a provider who already has thousands of hdd in operation buys considerably cheaper than Jane and Joe.
Plus - and I know that this is somewhat "religious" - such a provider can go cheap and use 2nd hand drives, push his drives to the lifetime limit, etc. For such a provider it's merely a question of having tech hands and spare drives on location.
As for the hardware I'd do it differently.
From what I see 6TB hdds are the sweet spot currently. Concerning the base I agree with the type (but not necessarily the make and model) of what cociu has in mind. Having two boxen with 62 hdd bays each nicely matches the 600 TB + raid + nominal vs. true capacity.
Where I would save money is raid hardware. 124 raid 5 or 6 ports is serious money. So I'd do raid 5 (or if the customer pays, raid 6) in software using the 2 boxen with their 26xx cpus basically as smart drives cages (DAS/NAS). And I'd put plenty RAM in them.
The final element would be what could be called the front end, i.e. the box with 3 Gb bandwidth.
Btw, the really interesting question that this topic triggered in my (granted, somewhat weird) head is this: Do linux and/or the BSDs make use of hw. engines in the cpu, or do they just assume a plain common denominator x64?
Depending on from where one looks that is more or less true, yes. I, too, think that cociu fucked the storage vps thing up, beginning on the concept level (e.g. ovz, too big partitions, etc))
But still, someone utterly fucking up with a big storage special does not at all translate to him being generally incapable. And indeed there are thousands of HS customers (on non storage vps) who are quite happy sailors with HS.
And btw. I also feel that cociu gets a lot of punishment here because he is honest. Most others would play the corporate cover up PR games, point fingers, etc. cociu simply and plainly publicly says "We fucked up, sorry" and reports even gory details (like the never ending fsck).
But HS and cociu is not just those fucked up storage vps. They are also a provider with lots and lots of well deserved happy customers and a guy who chose the hard route by starting with his own DC right away (instead of renting colo like most do). Which often translates into insanely attractive pricing that others can't easily match.
Would you like to be reduced to only your failures? I don't think so. So I suggest to be fair to others, too. Yes, cociu fucked up one thing. But many others he did and does well and smart.
This is starting to feel like "everyone gets a prize for participating" zone.
I sincerely wonder your explanation for this part as well.
What part is confusing? Maybe rooting isn't translating right?
I hate seeing peoples businesses go under minus a select few people. It's always good to see people come up with new ideas, products, and such, especially if they work out well. I hope for his sake and his customers that he solves those storage issues and everyone's happy.
Francisco
I actually asked @bsdguy to explain that part.
Ah, I didn't see anything of his get copied so I was a bit confused :P
Francisco
Yeah, it looks a bit confusing. He quoted a sentence of yours and replied to some points of yours without quoting them so I copied one of the parts he didn't mention and asked for a explanation.
I've said what I had to say at that.
Withdrawn, I'm calm now.
ok you are the king , what is next ? because repeat the same comments for infinite i dont think is something constructive here. I have learn my lesson about storage , i have apologise , i have change almost 80% of electrical structure wich was the initial cause , a fuking fusile from the principal ups , i have make upgrade to our network , i have provide all vps finnaly +3 month like compensation , i have ask apologise and reconose our fault .... what i can do more ? I dont wish you any negative expericence but if you will happened something like this with almost 190 tb data i am curious how you will reacting . Also we will talk in 1 year about my storage offers and i am curious what you will tell in this point because yes i will still make such deals in the near future and i am sure will have many customers happy .
Also i have faill in storage vps not in all my services i think ...
Fair point (I think). Still, your focus should be on addressing the storage issues and getting these people fixed.
You need to remember, customers having issues with storage (and still having issues) see you posting trying to make more sales instead of making them happy. For some users, credit means nothing, they want a working service.
Many other hosts have been suspended on these forums for far less so you really shouldn't throw that away.
Francisco
we will talk in 1 year.
https://www.ovh.com/us/dedicated-servers/fs/160fs5.xml
Crazy amount of configuration
Why not?
@netomx Can Taco Hosting do this? How is that project anyways?
It's bad for sales
Oi, ye bastards. Stop hatin on Cociu, I am trying to close a deal.
EU law enforcement confiscates the entire hardware and can take years to return it. No compensation.
US as well - see Megaupload. Nothing returned, years raided.
No, not really - even the largest buyers only get a few % over the retail price.
Is that how long its going to take to fix your current customers?
Maybe. Can he receive 5 tacos a week as a gift?
http://www.webhostingtalk.com/showthread.php?t=1660249
OP may find this interesting