Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


What's the biggest mistake you've made on a server? - Page 6
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

What's the biggest mistake you've made on a server?

12346

Comments

  • I miss the "screen password" :/ haha

  • Mark_RMark_R Member
    edited August 2014

    @GoodHosting said:
    The biggest mistake I made was ever installing cPanel anywhere. Seriously, screw cPanel. Ours is down again, I'm transferring all the sites out to nginx as I write this post, cPanel can go burn in the hell it created for itself. I'm getting sick and tired of seeing the kernel screaming "CPSRVD NOT TAINTED, KILL EEET"

    You should consider the Interworx panel, its really great and affordable if you use Licensecard. I've used the interworx panel before and I really was confident about the support from the developers, if anything goes wrong they are there to help! On top of that the interworx panel has a remote assistance function, after enabling that the interworx team would be able fix problems instantly for you by accessing your server on approval.

    Interworx is more user-friendly with its interface aswel.

  • @Mark_R said:
    Interworx is more user-friendly with its interface aswel.

    Well, turns out that (at the moment) it's not cPanel's fault. Some angry little skiddy is "attacking" the webserver. https://db.tt/oAhjYgRL I'm very glad that OpenNebula supports Cloud Bursting, I can burst our cPanel up to AWS provisioned firewall or some other service that can tank and filter for us.

    Thanked by 1Mark_R
  • @GoodHosting said:
    Well, turns out that (at the moment) it's not cPanel's fault. Some angry little skiddy is "attacking" the webserver. https://db.tt/oAhjYgRL I'm very glad that OpenNebula supports Cloud Bursting, I can burst our cPanel up to AWS provisioned firewall or some other service that can tank and filter for us.

    Aren't you using OVH aka being behind their ddos shield? they don't filter out application focused attacks like ssyn?

  • BrianHarrisonBrianHarrison Member, Patron Provider

    @eastonch said:
    yum remove bash :(

    How was that one an accident? :) Pretty difficult to typo something like that!

  • @Mark_R said:

    Nope! And to top it off, it was OVH's servers doing the attacking. Once we cleaned up the iptables and kernel log on the VM, then rebooted it (with 32GB of RAM and 16 cores this time; so that it could front the extra 200 load created by 256 concurrent whmcs requests...) we found that the IP addresses hitting the server hardest were owned by ovh.net , and announced as their infrastructure.

    Thirty (30) emails went out to [email protected] (one for each abusing IP address, with full tcpdump logs and iptables / securelog entries to prove the attach) We'll see if they even care to fix it or respond.

    What a joke.

    Thanked by 1Mark_R
  • MichaelBuiMichaelBui Member
    edited August 2014

    rm -rf /*

    instead of

    rm -rf ./*

  • SaahibSaahib Host Rep, Veteran

    Few were quite hilarious..

    For me, updated wrong server and messed up all sites on it as it was running a special config .. took couple of hours to fix .

  • Updated inittab and instead of doing /sbin/init q bymistake I did init 1, and server went down in single user mode out of all connections...

  • forgot to block port 22...

  • MakenaiMakenai Member
    edited April 2015
    cp -R .* ../workdir/ 
    

    Didn't realise what it was doing until it was too late, filled the server with copy of itself and a few applications just killed themselves.

    Thanked by 1howardsl2
  • ehabehab Member

    setfacl -R -m someUser:someUser:rwx /etc

    don't try this at home.

  • Hired a person from LET to be a tech, asked him to remove clients with no VPS from the server only to have him remove all clients from the server.

  • ehabehab Member

    @Clancoms said:

    then what happened?

  • @ehab said:
    then what happened?



    Thank god for backups

  • ehabehab Member

    :) yeah thank god for backups and everything.

  • emgemg Veteran

    A long time ago ... in a galaxy far far away.

    The "purge" command in the VAX VMS operating system means, "Keep the latest version of each file in this directory and delete any old versions." It is a relatively harmless command that can free up disk space and reduce clutter in directory listings. I was an experienced VAX VMS sysadmin.

    My first day on a new project, which used the iRMX86 operating system, I learned that the "purge" command in iRMX86 means, "Delete all files in the current directory and every subdirectory below it." :-o

  • howardsl2howardsl2 Member
    edited April 2015

    mv some-temp-file /dev/null

    System was hosed... DO NOT TRY THIS

  • Nick_A said: This... With no backups... Several weeks of Minecraft building gone down the drain.

    I had a similar experience before where I removed my server accidentally from the node but just in case anyone comes across such issue, I solved this by immediately restoring the volume using the 'vgcfgrestore' command - then used testdisk to save the data

  • rokokrokok Member
    edited April 2015

    Run dd porn test

    let it idle

    And forgot i paid yearly for that :/

    Thanked by 2howardsl2 yomero
  • creepcreep Member

    biggest mistake i did on a server is i used centos instead of debian/ubuntu. i cried 30 days for commiting that sinful mistake.

  • MikePTMikePT Moderator, Patron Provider, Veteran

    Biggest mistake I've ever done on a server was reinstalling it without getting backups first. That ended up with the blog down, no backups, and from a really well-known mobile App.

    Good thing the staff was able to a backup of the blog posts.

  • cassacassa Member

    Removing a Steam game

  • cociucociu Member

    Disable all USB ports, and i use a keyboard with usb cable ..... LOL

  • err.. I reinstalled wrong server in solusvm. I shutdown my laptop and havent power it on for over a week. ('-_-)

  • LicensecartLicensecart Member
    edited April 2015

    I did what you said rm -f /* instead of rm -f ./*

    Thankfully for me the client was installing and setting up and I didn't loose anything important.

  • BotoXBotoX Member

    Run the OS off a flash drive, it has lifetime warranty, I'm running on the third flash drive now. (Transcend JetFlash 780 32GB)

    Encrypt backups, reinstall and forgot the key >_>
    (Good that I still had backups of /etc on another server, saved me some work heh)

  • J1021J1021 Member

    I've deleted authorized_keys without noticing and then disconnected from the dedicated server.

  • RalliasRallias Member
    edited April 2015

    I've broken networking several times. That's why from some arbitrary point in the past, I've done the following in a screen.

    ovs-vsctl add-br br0;  ovs-vsctl add-port br0 p4p1; service networking restart; sleep 300; cp /etc/network/interfaces.old /etc/network/interfaces; ovs-vsctl del-port br0 p4p1; ovs-vsctl del-br br0; service networking restart;
    

    I can easily CTRL+C if it succeeded, but if I fucked up... well then it reverts after 5 minutes.

    Oh... and then there's the time I left the install.sh file in the root directory... I didn't think that some dimwit would run the script again... its sole purpose WAS to install 3 customer-servicing VMs.

  • deadbeefdeadbeef Member
    edited April 2015

    rm -rf / instead of ./ on the NFS server, then franticly pressing ctrl-c.

Sign In or Register to comment.