Why does E3 SSD dedi seem faster than Ryzen 7950X NVMe 2 vCore VPS at `sha256sum -c`?

Comments

• CharityHost_org Member

  September 2024 edited September 2024

  @Not_Oles said:

  @jsg said: you seemed to have completely ignored my serious comment

  Nope! I appreciate your serious comments! I thanked your serious comment to which you refer. You can look and see. My thanks is there for you and for everyone to see.

  It's clear that you understand these topics about RAM and disk operations very well! I believe you understand more deeply, more thoroughly, and more comprehensively than I understand.

  @jsg said: What you need is 2 factors, (a) lots of RAM, and (b) nothing but the bare OS running (and of course your SHA program).

  It's Low End here. I was trying to see what I could do on an inexpensive VPS compared to my E3 dedicated server. I discovered what could be described as a "limitation" of the VPS. With help from the kind provider and from other LET members, I compared what happened when we increased the VPS cores and RAM. I also made an equivalent spec VPS on the E3 and compared that VPS' performance.

  There's more brewing behind the scenes, because, with even more help from the kind provider, it looks like I might be going to learn a little about certain parameters and how these differ between Debian and Ubuntu default installs. Probably, we're also going to compare with an Intel E5 CPU VPS. I still might post more here if, as, and when I find out more.

  So this has been a great thread for me -- thanks to you, @jsg -- and thanks to other members -- and thanks to the awesome Provider of this VPS. I hope this has been a good thread for all of you guys too! Best wishes @jsg! Best wishes, everyone!

  @Not_Oles
  I can give you a trial VDS 2 if you want for 1 month so you can test with Intel Xeon CPU E5-2696 v3 @ 2.30GHz. PM me for the benching and testing.
• CharityHost_org Member

  September 2024

  And indeed, more RAM is very useful for buffers/cache.
• CharityHost_org Member

  September 2024

  @Not_Oles said:
  Thanks to everyone who has commented.

  Just now remembering that the E3 is running ext4 with LVM, whereas the Ryzen is running xfs.

  I haven't yet studied up on the effect of these filesystem differences. Ideas, please?

  Indeed it all matters, not just CPU, not just IO from disk, assuming it's not capped/throttled.
• dev_vps Member

  September 2024

  @CharityHost_org said:

  @Not_Oles said:
  Thanks to everyone who has commented.

  Just now remembering that the E3 is running ext4 with LVM, whereas the Ryzen is running xfs.

  I haven't yet studied up on the effect of these filesystem differences. Ideas, please?

  Indeed it all matters, not just CPU, not just IO from disk, assuming it's not capped/throttled.

  Do you offer true VDS with dedicated cpu core?

  Again, dedicated cpu core … not core HT marketing gimmick.
• zakkuuno Member

  September 2024

  I don't think this is a complex problem -- io should be enough in most shared environment. It's just that sha256sum is a simple piece of software. If you have a shared storage, try add a buffer for it so that it does not have to read block by block from the disk:
  mbuffer -m 512M -i test_file | /usr/bin/time -v sha256sum

  Thanked by 1Not_Oles
• Not_Oles Member, Patron Provider

  September 2024

  @zakkuuno

  I have to try mbuffer! Thanks for the suggestion!

  chronos@penguin:~$ apt show mbuffer
  Package: mbuffer
  Version: 20230301+ds1-1
  Priority: optional
  Section: utils
  Maintainer: Peter Pentchev <[email protected]>
  Installed-Size: 378 kB
  Depends: libc6 (>= 2.34), libgcrypt20
  Homepage: https://www.maier-komor.de/mbuffer.html
  Tag: admin::backup, role::program
  Download-Size: 57.1 kB
  APT-Sources: https://deb.debian.org/debian bookworm/main arm64 Packages
  Description: tool for buffering data streams
   The mbuffer tool is used to buffer data streams and show the I/O rate and
   summary to the user.  It is especially useful for writing backups to
   fast tape drives or streaming them over the network.  If used correctly,
   it can prevent buffer underruns and speed up the whole backup or
   transfer process.
  
  chronos@penguin:~$
  

  Thanked by 1zakkuuno
• dev_vps Member

  September 2024

  #!/bin/bash
  
  # Input file passed as the first argument
  INPUT_FILE="$1"
  
  # Check if the input file is provided and exists
  if [[ -z "$INPUT_FILE" || ! -f "$INPUT_FILE" ]]; then
    echo "Error: Input file not provided or does not exist."
    exit 1
  fi
  
  # Calculate total available memory in MB
  TOTAL_MEM=$(free -m | awk '/Mem:/ {print $2}')
  
  # Set buffer size to 25% of total memory, with a minimum of 512MB and a max of 4GB
  BUFFER_SIZE=$(awk -v mem="$TOTAL_MEM" 'BEGIN {buf=mem * 0.25; if (buf < 512) buf=512; else if (buf > 4096) buf=4096; printf "%.0fM", buf}')
  
  echo "Using buffer size: $BUFFER_SIZE"
  
  # Execute the command with dynamic buffer size and monitor resource usage
  mbuffer -m "$BUFFER_SIZE" -i "$INPUT_FILE" | /usr/bin/time -v sha256sum
  
  # Capture the exit code and handle errors
  EXIT_CODE=$?
  if [ $EXIT_CODE -ne 0 ]; then
    echo "Error: Command failed with exit code $EXIT_CODE."
    exit $EXIT_CODE
  fi
  
  echo "SHA-256 checksum calculated successfully."
  
• CharityHost_org Member

  September 2024

  @oplink said:
  My guess would be your VPS has no swap space, and your DEDI has swap space to use.

  How large is the file your trying to SUM?

  Thanks
  Ryan

  Good call Ryan! Somehow missed this. If the file is massive, the swap will help, otherwise if you have enough memory available, that would help more.

  Andre

  @Not_Oles said: sha256sum target file is 4.6 GB. The VPS has 4 GB RAM and zero swap space. The dedi has 64 GB of RAM and 4 GB swap space.

  Aha! That's it.
• CharityHost_org Member

  September 2024

  @Not_Oles I bet the dedi also had more available RAM than the file size at the time you ran the time on the checksum, and your VPS did not (and no swap either). I think @oplink nailed it.