Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


β€Ί HAZI.ro | Test for my stress level and my SysAdmin skills
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

HAZI.ro | Test for my stress level and my SysAdmin skills

FlorinMarianFlorinMarian Member, Host Rep

Hello!

It happened again that our second server (sv2) reached a performance drop of approximately 16 times compared to the base references.

Since this also happened 8 days ago and the server is out of warranty next month, we decided together with the company from which the server was bought to send it back to the UK for diagnosis and replacement of faulty components.

Our plan is based on the following steps:
1. We backup all VMs and containers hosted there
2. Upload the backups to the Cloud
3. Migrate the VMs to sv4.hazi.ro (within the available space, as sv4 is purely SSD and 90% of the affected VMs have HDD storage)
4. We purchase a new server if the repair of the old one takes too long and we restore the rest of the VMs involved.

For any question, do not hesitate to contact us at [email protected]

Thank you for your patience!

Thanked by 1ehab

Comments

  • This is like "santa barbara" soap drama. Ffs. Where is that disclaimer?!

    Thanked by 1NobodyInteresting
  • AndreiGhesiAndreiGhesi Member
    edited September 2022

    At this point you should just call it.... You have no idea what you are doing and for who know what fucking reasons you tell everybody that you don't know what you are doing.
    How do you expect people to buy services from a company that can't manage his servers?

    @yoursunny help your brother

    Thanked by 1adly
  • yoursunnyyoursunny Member, IPv6 Advocate
    • Cluelessβ„’ administrator! πŸ˜€ Florin frequently messes up! 😱
    • Hey! There is no HDD anymore! 🀩
    • You will get sudo within your VPS. πŸ™ˆπŸ™‰πŸ™Š
    • Delivery to UK might take a week! The Queen is dying! 😴
    • No warranty, to the extent permitted by applicable law. No refunds. Not for business use. Intended especially for computer learning and fun! πŸ€‘
    • White Hat stuff only, please! No copyright violations, please. πŸ‘
    • Florin tries to keep the server in good shape. Offline repairs are guaranteed! It's horrifying 😱 or amazing βœ¨πŸ’–βœ¨ how slow these repairs are! You send the server back, and -- within a month or two -- the vendor still hasn't fixed the machine.
    • Please make your own redundant, offsite backups! It's easy to download or sync or clone your backup to a safe place. Please also make sure that you actually can restore from your backups! Please think of your HAZI account as ephemeral! πŸ€¦β€β™‚οΈ
    • HAZI is an indie project of Florin with help from his awesome brothers. HAZI is neither a lowendtalk.com nor an OVH project. πŸ‘¨β€πŸ’»β€‹
    • The server seller can see your disk image, your naked push-up videos, and much other information. So, please do not put confidential information on the server. πŸ€”
  • jsgjsg Member, Resident Benchmarker

    At least he is honest and transparent.

    That said, @FlorinMarian: man, get your act together unless you want to be laughed off the stage!

  • FlorinMarianFlorinMarian Member, Host Rep

    @AndreiGhesi said:
    At this point you should just call it.... You have no idea what you are doing and for who know what fucking reasons you tell everybody that you don't know what you are doing.
    How do you expect people to buy services from a company that can't manage his servers?

    @yoursunny help your brother

    The message is addressed to my clients, not to the Romanians who have nothing better to do than to repeat the new word they have learned, "fucking" Romanian.

  • jackbjackb Member, Host Rep

    @FlorinMarian said:
    The message is addressed to my clients, not to the Romanians who have nothing better to do than to repeat the new word they have learned, "fucking" Romanian.

    Isn't that what email is for? Unless you're seeking help I don't know what this post would achieve.

    1. Migrate the VMs to sv4.hazi.ro (within the available space, as sv4 is purely SSD and 90% of the affected VMs have HDD storage)

    Hope there won’t be an increase in price too for the surprise SSD upgrade

  • DPDP Administrator, The Domain Guy
    edited September 2022

    @jackb said: Isn't that what email is for? Unless you're seeking help I don't know what this post would achieve.

    If I had to guess, it's probably an "inb4" clients/customers create their threads.

    Thanked by 1yoursunny
  • FlorinMarianFlorinMarian Member, Host Rep

    @jackb said:

    @FlorinMarian said:
    The message is addressed to my clients, not to the Romanians who have nothing better to do than to repeat the new word they have learned, "fucking" Romanian.

    Isn't that what email is for? Unless you're seeking help I don't know what this post would achieve.

    I already sent another e-mail in the morning to this category of customers, it would not be ok to send two e-mails a day because they would end up not even being read.

    @jmaxwell said:

    1. Migrate the VMs to sv4.hazi.ro (within the available space, as sv4 is purely SSD and 90% of the affected VMs have HDD storage)

    Hope there won’t be an increase in price too for the surprise SSD upgrade

    Absolutely not.

  • yoursunnyyoursunny Member, IPv6 Advocate

    People need to chill.
    My bro is doing his best.
    He's literally a kid starting his first "lemonade stand".

    You pay €4.99/month for the drama.
    The KVM is a free gift included with the drama.

    When I grow up I wanna be like Florin.

    @jmaxwell said:
    Hope there won’t be an increase in price too for the surprise SSD upgrade

    Absolutely not, until his two classmates think otherwise.

  • FlorinMarianFlorinMarian Member, Host Rep

    Unfortunately, making the backups would have taken much too long (3-4 days considering that the data is read at 120MB/s) and besides that, I have noticed that those customers who have two disks attached, lose one of them , being omitted by backups.

    As long as I negotiate through the ticket system with those clients to replace the servers with two HDDs with RAID10 servers with a single HDD, all services are online.

    A new hypothesis I am going with is that by trying to reduce energy consumption I have limited the performance of the server altogether. More clearly, before the problems appeared, my server was consuming 240-270W and now, after I changed the algorithm by which the server's power is scaled, the performances are as below with power consumption ALWAYS > 330W.

    root@nextcloud:~# curl -sL yabs.sh | bash
    # ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
    #              Yet-Another-Bench-Script              #
    #                     v2022-08-20                    #
    # https://github.com/masonr/yet-another-bench-script #
    # ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
    
    Thu Sep  8 18:05:50 UTC 2022
    
    Basic System Information:
    ---------------------------------
    Uptime     : 0 days, 0 hours, 41 minutes
    Processor  : Intel Core Processor (Haswell, no TSX)
    CPU cores  : 1 @ 2299.998 MHz
    AES-NI     : βœ” Enabled
    VM-x/AMD-V : ❌ Disabled
    RAM        : 3.8 GiB
    Swap       : 0.0 KiB
    Disk       : 1.9 TiB
    Distro     : Ubuntu 22.04.1 LTS
    Kernel     : 5.15.0-47-generic
    
    fio Disk Speed Tests (Mixed R/W 50/50):
    ---------------------------------
    Block Size | 4k            (IOPS) | 64k           (IOPS)
      ------   | ---            ----  | ----           ----
    Read       | 126.48 MB/s  (31.6k) | 1.92 GB/s    (30.0k)
    Write      | 126.81 MB/s  (31.7k) | 1.93 GB/s    (30.2k)
    Total      | 253.29 MB/s  (63.3k) | 3.85 GB/s    (60.2k)
               |                      |
    Block Size | 512k          (IOPS) | 1m            (IOPS)
      ------   | ---            ----  | ----           ----
    Read       | 6.34 GB/s    (12.3k) | 7.89 GB/s     (7.7k)
    Write      | 6.68 GB/s    (13.0k) | 8.42 GB/s     (8.2k)
    Total      | 13.02 GB/s   (25.4k) | 16.32 GB/s   (15.9k)
    
    iperf3 Network Speed Tests (IPv4):
    ---------------------------------
    Provider        | Location (Link)           | Send Speed      | Recv Speed
                    |                           |                 |
    Clouvider       | London, UK (10G)          | 1.24 Gbits/sec  | 489 Mbits/sec
    Online.net      | Paris, FR (10G)           | 1.66 Gbits/sec  | 357 Mbits/sec
    Hybula          | The Netherlands (40G)     | 1.23 Gbits/sec  | 15.2 Mbits/sec
    Uztelecom       | Tashkent, UZ (10G)        | busy            | busy
    Clouvider       | NYC, NY, US (10G)         | 743 Mbits/sec   | 12.6 Mbits/sec
    Clouvider       | Dallas, TX, US (10G)      | 697 Mbits/sec   | 14.9 Mbits/sec
    Clouvider       | Los Angeles, CA, US (10G) | 734 Mbits/sec   | 9.98 Mbits/sec
    
    Geekbench 5 Benchmark Test:
    ---------------------------------
    Test            | Value
                    |
    Single Core     | 571
    Multi Core      | 571
    Full Test       | https://browser.geekbench.com/v5/cpu/17137394
    
    
    

    I am considering the purchase of a new physical server to replace the current server as it would cost me approximately 2000 euros, which a year ago cost 3500 euros.

    I will keep you updated.

    All the best, Florin.

  • AlexBarakovAlexBarakov Patron Provider, Veteran
    edited September 2022

    Why don't you start by resetting all the BIOS settings to default and setting the power profile to high performance, without doing any of your manual tweaks? There's no real way to reduce power usage without affecting performance. Set the tuned-adm profile to throughput-performance as well, inside the OS. And not keeping spares (be it complete servers) on-site will only cost you more in the long run.

    Oh and an year ago - 3500EUR used to buy a good EPYC CPU, compatible mobo, chassis, RAM and NVMes. You're severely overpaying for old-generation hardware.

    Good luck.

  • risharderisharde Patron Provider, Veteran

    16 times slower makes me wonder what on earth could cause such a performance degredation. I've only ever seen bad disk affect servers like that so curious really to know what else could cause such events.

  • @risharde said:
    16 times slower makes me wonder what on earth could cause such a performance degredation. I've only ever seen bad disk affect servers like that so curious really to know what else could cause such events.

    I agree. I don't think it's hardware related as it happened in two different servers.

    At what capacity are the disks?

  • @CyberCr33p said:

    @risharde said:
    16 times slower makes me wonder what on earth could cause such a performance degredation. I've only ever seen bad disk affect servers like that so curious really to know what else could cause such events.

    I agree. I don't think it's hardware related as it happened in two different servers.

    At what capacity are the disks?

    To be honest. My take is I think his power-saving methods by customizing his bios configurations may be fucking with the performance problem. I don't recall (as in I don't want to scroll back up there), but if he's tried resetting the bios and giving that a whirl.

    Thanked by 2yoursunny ralf
  • FlorinMarianFlorinMarian Member, Host Rep
    edited September 2022

    @risharde said:
    16 times slower makes me wonder what on earth could cause such a performance degredation. I've only ever seen bad disk affect servers like that so curious really to know what else could cause such events.

    Tested individually with fio, all disks seemed ok.

    @HalfEatenPie said:

    @CyberCr33p said:

    @risharde said:
    16 times slower makes me wonder what on earth could cause such a performance degredation. I've only ever seen bad disk affect servers like that so curious really to know what else could cause such events.

    I agree. I don't think it's hardware related as it happened in two different servers.

    At what capacity are the disks?

    To be honest. My take is I think his power-saving methods by customizing his bios configurations may be fucking with the performance problem. I don't recall (as in I don't want to scroll back up there), but if he's tried resetting the bios and giving that a whirl.

    I have not reset and I do not consider resetting the bios with the customer data stored inside.
    Resetting the bios also automatically changes some parameters related to the LSI controller and at the same time the IP configuration of the IPMI.
    I don't think that in addition to the already existing problems, it would be good for me to stay with a non-bootable server to which I have no access, and the next day when I would have received access to the server again, I would find that the partitions were corrupted by the magnificent idea of to do a factory reset :) (I speak from experience, I experienced it with my first physical server that I used for learning purposes)

    Regarding energy consumption..later I understood the fact that between a Dell with two 145W CPUs but with only 6 SSDs it consumes 90W without hosted VMs but instead an HP with 16 drives (12 HDDs, 4 SSDs) consumes 330W..even if the CPUs have only 135W.
    The difference comes from fans that are at 2-3000 rotations on the Dell and 12000 on the HP.
    Regarding the server costs, they are high, but it was a main requirement for me that it be 1U because the data center where the servers are hosted charges 10 euros + VAT for each unit and also another 10 euros for every 100W from the source, regardless of consumption.
    I was afraid that I would double the price of the colocation due to the size of the source and the unit, but I ended up doing it due to the energy consumption.
    I confirm a popular saying: "Life doesn't teach you anything, life gives you lessons".

    Dell 15% CPU avg. usage:

    HP 7% CPU avg. usage:

Sign In or Register to comment.