Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › BuyVM Catastrophic Data Failure - All data lost on a node! - Page 3
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

BuyVM Catastrophic Data Failure - All data lost on a node!

13567

Comments

  • lurchlurch Member

    No luck for me only, did 30/30 power only cycle x2, then plugged into usb caddy

    and get this when i disconnect it

    [5436099.256609] sd 21:0:0:0: [sdg]  Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[5436099.256621] sd 21:0:0:0: [sdg] Sense not available.
[5436099.256747] sd 21:0:0:0: [sdg] Write Protect is off
[5436099.256761] sd 21:0:0:0: [sdg] Mode Sense: 00 00 00 00
[5436099.256886] sd 21:0:0:0: [sdg] No Caching mode page found
[5436099.256910] sd 21:0:0:0: [sdg] Assuming drive cache: write through
[5436099.257412] sd 21:0:0:0: [sdg] READ CAPACITY failed
[5436099.257420] sd 21:0:0:0: [sdg]  Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[5436099.257430] sd 21:0:0:0: [sdg] Sense not available.
[5436099.257477] sd 21:0:0:0: [sdg] Asking for cache data failed
[5436099.257493] sd 21:0:0:0: [sdg] Assuming drive cache: write through
[5436099.257510] sd 21:0:0:0: [sdg] Attached SCSI disk

    with it connected I get

    fdisk /dev/sdg
fdisk: unable to open /dev/sdg: No such device or address

    or

    fdisk /dev/sdg
fdisk: unable to read /dev/sdg: Invalid argument
  • NeoonNeoon Community Contributor, Veteran
    edited April 2018

    Well, the main cause of divorce is marriage.

    Same with drives, they die at some point.

    Thanked by 4quick FHR cassa Janevski
  • williewillie Member

    Were these 850 EVO drives or PRO? Were there any issues with power failures while the drives were active? What model of Intel NVMe drives will you be using? Don't want to distract you from restoring, just wondering.

  • HarambeHarambe Member, Host Rep

    @lurch said:
    No luck for me only, did 30/30 power only cycle x2, then plugged into usb caddy

    >

    Bummer. Sounds like you just got a DOA SSD. Could leave it plugged in for a longer period of time and see, but just sounds like it's toast.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @willie said:
    Were these 850 EVO drives or PRO? Were there any issues with power failures while the drives were active? What model of Intel NVMe drives will you be using? Don't want to distract you from restoring, just wondering.

    EVO's.

    Never, ever, had a power issue on that node or anything of the likes. We've rebooted the node in the past for kernel updates and such without issue.

    We're using Intel P3600's and things like that. They aren't as fast as the 3700 series but it's shared hosting, we don't need 400,000 IOP writes. Very high endurance which is nice.

    Francisco

  • datanoisedatanoise Member

    The 850 use 3D NAND, the 840 old school MLC NAND. Less chips, less PCB, cheaper to produce. If the base if one of those 3D nand chips fails, your disk is dead. If you reached the maximum data written for this component, it's likely that all disks will die at the same time, or almost. (On Intel 520 - using real and good MLC - the data is compressed on the fly to write less on the chips and it is supposed to be able to survive a dead chip, at least from what I read, didn't try it myself: all SSDs aren't created equal...)

    Thanked by 1PandaRain
  • randvegetarandvegeta Member, Host Rep

    This is why our raid arrays use different brand/model drives. With that comes other issues though.

    Thanked by 2Falzo pullangcubo
  • NickNick Member, Patron Provider

    randvegeta said: This is why our raid arrays use different brand/model drives. With that comes other issues though.

    Do you run into a lot of stability issues there?

  • AidanAidan Member

    @randvegeta said:
    This is why our raid arrays use different brand/model drives. With that comes other issues though.

    Or just use drives from different batches/with different wear levels - though that's easier said than done.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    randvegeta said: This is why our raid arrays use different brand/model drives. With that comes other issues though.

    Not a terrible idea.

    I've made design changes to this node which will allow me to likely take deduplicated block based backups which I'll do along side Jet.

    As I said, so far only the 850 1TB's have given this issue.

    Francisco

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    datanoise said: The 850 use 3D NAND, the 840 old school MLC NAND. Less chips, less PCB, cheaper to produce. If the base if one of those 3D nand chips fails, your disk is dead.

    I've seen plenty of discussion on forums of Samsung Enterprise SSD's doing the same thing (the PM series) so I'm not going to rush to blame TLC, especially given my 840s' are happy clams.

    Francisco

  • TommySRVTommySRV Member

    last chance of rescue is ddrescue

  • doghouchdoghouch Member
    edited April 2018

    @Inded_Hosting said:
    Well is the best always to keep backups from your side also to keep a backup of the backup too if is possible.

    You’re almost at 15 posts!

    Thanked by 6Aidan jetchirag ralph kingjj Falzo Nomad
  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @TommySRV said:
    last chance of rescue is ddrescue

    If we could see the drives but we can't, they never properly bootup.

    If this was just a case of needing dd rescue i would've completed that last night.

    Francisco

  • Inded_HostingInded_Hosting Member

    @doghouch said:

    @Inded_Hosting said:
    Well is the best always to keep backups from your side also to keep a backup of the backup too if is possible.

    You’re almost at 15 posts!

    ... http://prntscr.com/j0ylcs

  • cociucociu Member

    bad luck ... the shit is came when no one is espected.

    Thanked by 1cassa
  • vimalwarevimalware Member

    @cociu said:
    bad luck ... the shit is came when no one is espected.

    This could be the official sysadmin t-shirt

    Thanked by 3Francisco quick cassa
  • lurchlurch Member

    @cociu said:
    bad luck ... the shit is came when no one is espected.

    fsck?

  • jetchiragjetchirag Member
    edited April 2018

    @Inded_Hosting said:

    @doghouch said:

    @Inded_Hosting said:
    Well is the best always to keep backups from your side also to keep a backup of the backup too if is possible.

    You’re almost at 15 posts!

    ... http://prntscr.com/j0ylcs

    http://storage9.static.itmages.com/i/18/0404/h_1522877090_1390716_6521e8e1e4.png

    I dont really (think/know) you are doing it, heck haven't even checked your profile but couldn't resist

    Thanked by 1TommySRV
  • Inded_HostingInded_Hosting Member
    edited April 2018

    @jetchirag said:

    @Inded_Hosting said:

    @doghouch said:

    @Inded_Hosting said:
    Well is the best always to keep backups from your side also to keep a backup of the backup too if is possible.

    You’re almost at 15 posts!

    ... http://prntscr.com/j0ylcs

    http://storage9.static.itmages.com/i/18/0404/h_1522877090_1390716_6521e8e1e4.png

    I dont really (think/know) you are doing it, heck haven't even checked your profile but couldn't resist

    @jetchirag said:

    @Inded_Hosting said:

    @doghouch said:

    @Inded_Hosting said:
    Well is the best always to keep backups from your side also to keep a backup of the backup too if is possible.

    You’re almost at 15 posts!

    ... http://prntscr.com/j0ylcs

    http://storage9.static.itmages.com/i/18/0404/h_1522877090_1390716_6521e8e1e4.png

    I dont really (think/know) you are doing it, heck haven't even checked your profile but couldn't resist

    @jetchirag said:

    @Inded_Hosting said:

    @doghouch said:

    @Inded_Hosting said:
    Well is the best always to keep backups from your side also to keep a backup of the backup too if is possible.

    You’re almost at 15 posts!

    ... http://prntscr.com/j0ylcs

    http://storage9.static.itmages.com/i/18/0404/h_1522877090_1390716_6521e8e1e4.png

    I dont really (think/know) you are doing it, heck haven't even checked your profile but couldn't resist

    And where do you see spam here ? I only said to take backups which part of the line contained spam ?

  • TommySRVTommySRV Member

    @Francisco Did you tried to put these disks to another server? About a year ago I had a similar situation with my wife's laptop disk. I put the disk into my laptop but it didn't help. Then I switched the disk to my PC and I was so supriesed- disk appeared.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @TommySRV said:
    @Francisco Did you tried to put these disks to another server? About a year ago I had a similar situation with my wife's laptop disk. I put the disk into my laptop but it didn't help. Then I switched the disk to my PC and I was so supriesed- disk appeared.

    Earlier in this thread I mentioned that we tried no less than 4 different servers and around 6 different chipsets/HBA's to try to get it going.

    Francisco

  • jetchiragjetchirag Member

    Inded_Hosting said: And where do you see spam here ? I only said to take backups which part of the line contained spam ?

    As I said in small text, I had no such intention!

  • doghouchdoghouch Member

    @Inded_Hosting It's a running joke here that all new hosts rush and create 15 crappy posts to meet the requirements. (i.e. I have nothing against you.)

  • Inded_HostingInded_Hosting Member
    edited April 2018

    @doghouch said:
    @Inded_Hosting It's a running joke here that all new hosts rush and create 15 crappy posts to meet the requirements. (i.e. I have nothing against you.)

    But you do know that am only a shared hosting provider and i cant do anything even if i reach that number don't you ?

  • somewhatwindysomewhatwindy Member
    edited April 2018

    @doghouch Just take a look at his profile.

    Edit: there we go, permanent ban is now secured.

    Thanked by 1doghouch
  • Inded_HostingInded_Hosting Member

    @somewhatwindy said:
    @doghouch Just take a look at his profile.

    There is a url at my signature put the glasses on and check it...

  • TommySRVTommySRV Member
    edited April 2018

    @Francisco I'm sorry, I didn't notice it. From my side I only wish you good luck. If you will able to run with these disks, please share what You did.

  • randvegetarandvegeta Member, Host Rep

    @Aidan said:

    @randvegeta said:
    This is why our raid arrays use different brand/model drives. With that comes other issues though.

    Or just use drives from different batches/with different wear levels - though that's easier said than done.

    When I order new drives from our vendor, can't really specify different batches.

    Much easier to just order lots of different models/brands.

    It also makes it easier to identify dodgy disks since the serial number is often obscured by the disk cradle.

    Thanked by 2Aidan Aidan
  • corbpiecorbpie Member

    It was buyshared not buyvm champ

Sign In or Register to comment.