About your experience with HA storage (ceph)

Comments

• hyperblast Member

  April 2024

  @crunchbits said:

  @hyperblast said:

  @Athorio said:

  @hyperblast said:

  @host_c said:
  @hyperblast

  Not even bare mental PCI-EX Gen 4 NVME can do 8G @ 1 MB. not to mention a VPS. Cached.

  I have to admit, well done, respect.

  the honour is due: @dataforest @Avoro
  maybe the representatives of @dataforest can comment on this?! thanks!

  dude.. ceph is replicated over network, only 1/n is stored locally. Mostly 10G sometimes 40G is used.

  considering this facts, even without replication you can only transfer with 40G around max 4GB/sek, its physics.. nothing more..

  dude, so that means that yabs delivers incorrect values?

  Nah, it's just easy to "game" with RAM cache. There's nothing wrong with that (you should leverage it), but @Athorio is right too: 18GB/s = ~144Gbps, meaning a pair of 100G NICs are dedicated to the storage network alone and it would have to be fairly large ceph cluster to even hit those speeds (a lot of osd's).

  With ZFS and writeback/RAM you can hit 18-22GB/s on HDDs in YABS It's just a tool, one of many, to give you a quick idea. Personally, I think YABS is best used comparing your VM to itself overtime or maybe for extreme outliers (i.e. a 100 GB6 single core score confirming the CPU steal you see to be true).

  anyway dude, do you read private messages here on let?