Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Are servers based on consumer grade hardware reliable enough for production?

Are consumer grade servers suitable for use in production?
The CPU itself is likely to be very reliable, but most mainboards used for consumer grade CPUs seem to be of a different quality than enterprise mainboards

Comments

  • MikeAMikeA Patron Provider, Veteran
    edited November 9

    doubt much of a difference. the main problems usually stem from prosumer type NVMe drives for me. but I've also had expensive hardware, or things like Supermicro boards, be very problematic. it probably doesn't matter what you buy, it just comes down to luck. you pay more to have a easy replacement, so enterprise don't complain about problems because they get them replaced easily (well, maybe not now days, unless you are OpenAI or Anthropic or Elon).

    Also, having proper cooling is a huge thing. If components are cooking, shit hits the fan.

    Thanked by 1iriska
  • AdvinAdvin Member, Host Rep
    edited November 9

    I've seen some wild issues on EPYC but it has overall been more stable/predictable than Ryzen. We've had some really weird issues on Ryzen with virtualization in the past and often times we wouldn't have any idea as to what was going on (e.g., random freezing, reboots without explanation, etc). At the same time, we've had issues with EPYC servers in the past.

    We do have a bunch of Ryzen servers in our fleet that haven't had issues since day 1, and we are comfortable running some production services on Ryzen processors. It's somewhat of a rarity for there to be issues as the platform has matured, but when there are, they are generally just harder to diagnose/usually some much more obscure issue.

    I feel like hardware has gone downhill in terms of stability with how these platforms are being pushed to their limits (e.g., DDR5 at extreme speeds, high core processors, insane amounts of PCIe, etc).

    Thanked by 3oloke TrendyJack forest
  • kukara4kukara4 Member, Patron Provider

    I would say it’s more about architecture. Anything with a single copy is not production, even on enterprise servers.

    Thanked by 1MikeA
  • waqidwaqid Member

    Epyc has more testing in fabs hence more stable than ryzen

  • Agree with kukara4 that redundancy matters more than the badge, but a few concrete differences are worth checking before you put something important on a consumer box:

    • ECC: lots of Ryzen boards accept unbuffered ECC but don't always report corrected errors. If the host advertises ECC, check dmesg | grep -i edac (or edac-util -v) to see if it's actually active.
    • NVMe: consumer drives usually have no power-loss protection and lower TBW. Fine for web/app servers, but for databases doing lots of fsync, enterprise drives are noticeably faster and safer. smartctl -a /dev/nvme0 shows wear ("Percentage Used"), so you can see what you were given.
    • Out-of-band management: most consumer boards have no IPMI/BMC, so a hard hang means waiting for the provider's KVM or remote hands. Worth asking what their response time is.
    • Burn-in: run memtest and a stress test on delivery; most hardware faults show up in the first days.

    If the app can survive one box dying (backups plus a second node or a quick restore), consumer hardware at a lower price is usually a good trade.

    Thanked by 1kukara4
  • One thing nobody mentioned yet, most consumer builds run a single PSU, enterprise boxes usually have two. a dead PSU is one of the more common failures, and on a single PSU box that's just downtime until someone walks over and swaps it

  • For over a decade, a large chunk of both OVH and Hetzner was exactly that: consumer hardware, desktop CPUs and no ECC. But some custom cooling, in case of OVH.
    And plenty of people used it for production with no extreme issues.

  • mp11mp11 Member

    google was build on it.

  • layer7layer7 Member, Host Rep, LIR

    Hi,

    entirely depends on the application and how important is uptime and correctness of data.

    You are running a financial empire on a database that should better ensure correctness of data on the software and hardware side? Maybe a raspberryPi is the wrong choice...

    You are running a PiHole or datagrave? Maybe the AMD Epyc / Intel Xeon will waste your money.

    Just like already written from others....
    ...Consumerhardware has usually less mechanics to protect from error ( (REG)ECC RAM / drive / raidcontroller Powerlossprotection ) and can usually not scale ( no multi CPU support, no 10++ RAM slots, ... ).

    But especially nowadays, when it comes to recent CPU's there might not be much internal differences between an AMD Ryzen and its AMD Epyc pendent. Same Socket...


    And yes, the whole early internet was build on desktop hardware. Somehow they survived and grew ;-)

    Thanked by 1rpqu
  • Speaking from personal experience here: the performance gap between consumer-grade and enterprise hardware can easily be 100x. What’s worse is that you won’t catch this with a quick YABS run. Definitely keep your guard up.

    https://lowendtalk.com/discussion/198912/weird-mariadb-slowness-and-high-iowait-on-htop-output/p1

    https://hsu.cy/2022/03/mac-ssd-cheating/

    Thanked by 1newvps123
  • @PingKing said:
    Speaking from personal experience here: the performance gap between consumer-grade and enterprise hardware can easily be 100x. What’s worse is that you won’t catch this with a quick YABS run. Definitely keep your guard up.

    https://lowendtalk.com/discussion/198912/weird-mariadb-slowness-and-high-iowait-on-htop-output/p1

    https://hsu.cy/2022/03/mac-ssd-cheating/

    is there an alternative to yabs for disk benchmarking?

  • rpqurpqu Member

    @newvps123 said:

    @PingKing said:
    Speaking from personal experience here: the performance gap between consumer-grade and enterprise hardware can easily be 100x. What’s worse is that you won’t catch this with a quick YABS run. Definitely keep your guard up.

    https://lowendtalk.com/discussion/198912/weird-mariadb-slowness-and-high-iowait-on-htop-output/p1

    https://hsu.cy/2022/03/mac-ssd-cheating/

    is there an alternative to yabs for disk benchmarking?

    I got vibecoded one, https://iolite.st
    Beware, running it will reduce SSD health

  • Not when it comes to storage or memory, in my opinion.

  • @emiraykoz said:
    Not when it comes to storage or memory, in my opinion.

    does using enterprise datacenter nvme drives with consumer cpus solve the storage issue?

  • forestforest Member
    edited November 10

    Most "enterprise" hardware is about greater visibility into device health and performance and features that you wouldn't necessarily need on a typical consumer device. More sophisticated diagnostics, features that improve scalability, etc.

  • @newvps123 said:

    @emiraykoz said:
    Not when it comes to storage or memory, in my opinion.

    does using enterprise datacenter nvme drives with consumer cpus solve the storage issue?

    Well yeah, enterprise drives are more reliable.

  • @newvps123 said:
    is there an alternative to yabs for disk benchmarking?

    yabs just runs fio for the disk part anyway, so you can run fio yourself with a bigger file and a longer runtime. that's where the cheap consumer drives fall apart, once the cache fills up

  • @newvps123 said:
    Are consumer grade servers suitable for use in production?
    The CPU itself is likely to be very reliable, but most mainboards used for consumer grade CPUs seem to be of a different quality than enterprise mainboards

    production? for a bank or a local football club? The difference in solution is a factor 1000000.

Sign In or Register to comment.