Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


What is Cheapest Server with PCIE 4.0 Slots?
New on LowEndTalk? Please Register and read our Community Rules.

What is Cheapest Server with PCIE 4.0 Slots?

laobanlaoban Member

Hello,
Any recommendation on server or motherboard with PCIe 4.0 slots and bifurcation support?
I am thinking to put as many as fast NVMe M.2 disks to build a Ceph cluster.
I actually don't know whether this is a good idea or not, any suggestions and advice are very appreciated.
Thank you very much...

Comments

  • Thank you for your recommendation, but I want to buy the machines and build the Ceph cluster in my own place.

  • CiprianoOscarCiprianoOscar Member, Provider

    @laoban said:

    Thank you for your recommendation, but I want to buy the machines and build the Ceph cluster in my own place.

    Ah ok sorry, I didn't understand.

    Simply buy the parts and assemble it yourself

    Thanked by 1laoban
  • @laoban said:
    Hello,
    Any recommendation on server or motherboard with PCIe 4.0 slots and bifurcation support?
    I am thinking to put as many as fast NVMe M.2 disks to build a Ceph cluster.
    I actually don't know whether this is a good idea or not, any suggestions and advice are very appreciated.
    Thank you very much...

    I recently bought a used SuperMicro from eBay along with a cheap Epyc CPU. Has like 6 PCIe slots and a couple M.2's.

    I did see pictures of a MB that had like 11 onboard m.2 slots and a couple of PCIe slots but it was custom and not retail for loose sales. I forget where I saw that.

    How many are you looking to get per server? Needing more than 8 per board probably narrows it down a lot.

    Thanked by 1laoban
  • @TimboJones said:

    @laoban said:
    Hello,
    Any recommendation on server or motherboard with PCIe 4.0 slots and bifurcation support?
    I am thinking to put as many as fast NVMe M.2 disks to build a Ceph cluster.
    I actually don't know whether this is a good idea or not, any suggestions and advice are very appreciated.
    Thank you very much...

    I recently bought a used SuperMicro from eBay along with a cheap Epyc CPU. Has like 6 PCIe slots and a couple M.2's.

    I did see pictures of a MB that had like 11 onboard m.2 slots and a couple of PCIe slots but it was custom and not retail for loose sales. I forget where I saw that.

    How many are you looking to get per server? Needing more than 8 per board probably narrows it down a lot.

    Was it 1U server? Would you like to share the link?
    I think 8 per server is enough for initial setup.

  • lentrolentro Member, Provider

    @TimboJones said: SuperMicro

    OP seems to want PCIE 4.0, and from what I've seen, most used SuperMicro EPYC motherboards only do PCIE 3.0. But, PCIE 3.0 should be enough for M.2 (16 GB/s if i remember correctly), so maybe @laoban you don't need to worry about the PCIE version being 4.0 and can broaden your search?

    Thanked by 1laoban
  • @lentro said:

    @TimboJones said: SuperMicro

    OP seems to want PCIE 4.0, and from what I've seen, most used SuperMicro EPYC motherboards only do PCIE 3.0. But, PCIE 3.0 should be enough for M.2 (16 GB/s if i remember correctly), so maybe @laoban you don't need to worry about the PCIE version being 4.0 and can broaden your search?

    I have tried it with PCIE 3.0, it can work good enough indeed. But I am looking for potentially faster speed.

  • what m.2 NVMe will be used?

  • @cybertech said:
    what m.2 NVMe will be used?

    Ceph OSD

  • @laoban said:

    @TimboJones said:

    @laoban said:
    Hello,
    Any recommendation on server or motherboard with PCIe 4.0 slots and bifurcation support?
    I am thinking to put as many as fast NVMe M.2 disks to build a Ceph cluster.
    I actually don't know whether this is a good idea or not, any suggestions and advice are very appreciated.
    Thank you very much...

    I recently bought a used SuperMicro from eBay along with a cheap Epyc CPU. Has like 6 PCIe slots and a couple M.2's.

    I did see pictures of a MB that had like 11 onboard m.2 slots and a couple of PCIe slots but it was custom and not retail for loose sales. I forget where I saw that.

    How many are you looking to get per server? Needing more than 8 per board probably narrows it down a lot.

    Was it 1U server? Would you like to share the link?
    I think 8 per server is enough for initial setup.

    I just bought the mobo and cpu, not chassis. But is PCIe 3, not 4. But searching eBay for used gear is still my recommendation.

    Thanked by 1laoban
  • AdvinAdvin Member, Provider

    5600X + X570 mobo is probably the cheapest Gen4 setup

    Thanked by 1laoban
  • PulsedMediaPulsedMedia Member, Provider

    @TimboJones said: I recently bought a used SuperMicro from eBay along with a cheap Epyc CPU. Has like 6 PCIe slots and a couple M.2's.

    check those SC Motherboard M.2s ;) Splits the bandwidth in half of both are being used, you need their low profile 2xM.2 card to utilize more than 1x at full speed sadly.
    Fell into this trap too when building a custom server for customer and first time using SC EPyC motherboard couple years back.

    Most don't even seem to have 2x M.2 on board. I think we will go back to using ASRockRack or perhaps try Gigabyte MZ0-AR0 for our next EPyC server.

    @TimboJones said: I did see pictures of a MB that had like 11 onboard m.2 slots and a couple of PCIe slots but it was custom and not retail for loose sales. I forget where I saw that.

    Wow, that would be sweet sight to see at least, even if not purchaseable! The PCI-E x16 to 4xM.2 are full profile and cost like 60€ a piece.
    If we could get hands on one or two we could do a special one off dedi offering with them, just for the lols :)

    @laoban X570 server mobo + 5600X can get you to 6x M.2, 2 on board, 4 on a 4xM.2 card. Other than that your next bet option is going for EPyC. 1st CPUs can be had for under 300$ these days, and motherboards from ~600$. So actually the EPyC is not that bad comprated to X570 server mobo + 5600X :O Just picked up a bunch of 24 core 1st gen EPyCs at 290€ a piece. But 5600X is going to be insanely fast on single core in comparison.

    @laoban said: Ceph OSD

    what kind of ceph cluster are you doing? What's the use case here?

    Thanked by 1laoban
  • laobanlaoban Member
    edited January 16

    @PulsedMedia said:
    what kind of ceph cluster are you doing? What's the use case here?

    It is for VM storage. I plan to use minimal setup with 3 nodes.

  • PulsedMediaPulsedMedia Member, Provider
    edited January 16

    @laoban said:

    @PulsedMedia said:
    what kind of ceph cluster are you doing? What's the use case here?

    It is for VM storage. I plan to use minimal setup with 3 nodes.

    cool hence the proxmox?

    You need them all to be on same location, so you would be buying 3 of these same nodes or ... ?

    If you are looking for many, and willing to commit for 3+ months we can get you sorted out. Got a bunch of X570 servers being assembled now, not much effort to ditch the HDDs, and put in 4x M.2 card, thus getting 5 total on those boards (no second M.2 on motherboard)

    Thanked by 1laoban
  • @PulsedMedia said:

    @laoban said:

    @PulsedMedia said:
    what kind of ceph cluster are you doing? What's the use case here?

    It is for VM storage. I plan to use minimal setup with 3 nodes.

    cool hence the proxmox?

    You need them all to be on same location, so you would be buying 3 of these same nodes or ... ?

    Yes, all three nodes in same place, in my own place. I can add more nodes if needed later.
    Proxmox is cool, but I use libvirt for the virtualization.

  • ehabehab Member

Sign In or Register to comment.