New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Why server was offline - how to know it?
This discussion has been closed.
Comments
Sure, we call it "The system is hung." The VPS or whatever is running, drawing power, and the CPU is doing something (what??), but the system is non-responsive to input and there is no output. It can happen at startup or any other time. Operating systems are designed to prevent users and user level applications from triggering the system to hang, but there are always bugs.
Many people have written code that can can cause the system to hang. I have written plenty of code like that myself. In my case, I had to find and squish the bugs that caused it, of course.
By the way, I just tested the encrypted drive scenario on a VPS. I shutdown the VPS and the SolusVM status showed "Offline." After I clicked the "Boot" button in the SolusVM control panel, the Status quickly changed to "Online". The VPS was not really running yet. It would not respond to SSH or other input.
I had to open a console and enter the passphrase for the VPS to complete the boot process. The SolusVM status showed "Online" before and after that was done, but the VPS became responsive only after I entered the passphrase and allowed the boot process to complete.
Fair enough.
If you've looked at /var/log then you probably have all the info you can get.
You can look at dmesg (that's a command if you didn't know) to see the kernel messages but that's since boot - may be a hint if you see something in there but unlikely.
You can configure Linux (I assume that's the OS we're talking about) to dump if it panics, but you have to be able to analyze the dump files.
Did you just see a message in syslog that the system was booting - i.e., no message it was going down, everything's normal but the next message is from startup? Might be worth asking your provider if there was a hiccup.
A scenario I am all too familiar with (Maybe the first and only thing that came to mind, that's why my question :-)
Thx for the elaboration.
deleted
On some of my VPSes, I've experienced cases where I have literally nothing in the logs. Sometimes the host system will suddenly go down (power outage, forced shut down, etc) and hence won't write logs nor flush anything to disk.
Years ago, a free hosting company I was doing support for encountered something like this, with some servers at Layered Technologies. Not at the same time every day, but sporadically it'd restart. It turns out one of the techs tripped over a power cord (which caused the initial outage) and then didn't plug it back in properly, so sometimes it'd become partially dislodged.
That plus them accidentally replacing and formatting both hard drives instead of just one faulty one meant that we didn't really think positively of their techs any more, and moved somewhere else (Softlayer I think?)
I remember years ago phoning support for my new mobile phone that kept dropping wifi connections every couple of minutes. Obviously, I thought the phone was faulty and wanted it replaced.
Their customer service droid asked, "Have you rubbed the battery contacts with a rubber?"
I replied, "Of course not, why would I have done that?"
"Sorry, but we can't proceed with the next stage of the script unless you say you've done it."
"Can I just say I've done it and move on?"
"No"
I presumed this was all part of an elaborate scheme just to pass me on to someone else, because obviously removing the battery would mean I'd have to hang up.
So I hung up, and rubbed the battery contacts with a rubber as instructed.
And all the problems went away. So clearly it was in fact some kind of intermittent power issue from a dirty contact and there was sufficient capacitance for the CPU and screen etc to work fine during momentary power loss, but not enough for the wifi chipset.
I've started giving support people and their stupid scripts a bit more benefit of the doubt now, even though they're frustrating and sound ridiculous at the time.
Log files are your best option. Like everyone mentioned before
Sometimes its a mystery.
Had one KS1 loosing power, OVH support obviously clueless.
Some french baguette crumbs must have cut of the flow.
However without LOG, I would never known a french baguette had struck.
@vyas11
One fine day, your server is offline, you booted and its live. I am talking about this reboot.
someone set up you the bomb
All my unexpected issues have turned out to be bad RAM or PSU. I've never seen a failure for high resource except 0 bytes remaining type caused by PEBKAC. Those tend to be more obvious than mysterious.
Carpet in a server room? That's going to be bad times. "Secure area" with custodial access? That's going to be bad times. It's funny that having a really high quality UPS hid the problem as most UPS would trip from vacuum use on a small office UPS.
I intensely dislike these challenges from @TimboJones about the details of some brief experience I wrote about. He never says, "you made that up" or "you're lying", but he always calls into question some minor detail that make it feel like that is what he is implying. Timbo asked, so here are the specifics as I remember them from forty years ago:
I worked on the second floor of a large building. Part of my job involved coordinating with the leader of the group that "owned" the server. He was the one who told me about their mysterious server reboot problems.
The building was part of an enormous facility with many huge buildings and well over 10,000 employees coming and going every day. The multi-year project relied on hundreds of employees, who occupied most of the building. Nothing I mentioned was government classified. (I am resisting the urge to say that we stored our sensitive government classified documents at Mar-a-Lago.)
The server was in the basement of the building. It was in a typical computer room with a hard, raised floor made from large white/gray square removable tiles. The server was a DEC VAX and the UPS was a large cabinet nearby. They were not adjacent to each other, and there was other equipment in the room. The ordinary door to the room was protected with an ordinary cipher lock, the kind with black click buttons that you press in sequence. The team who worked in the room had access. So did a few others who worked with them, like me. I rarely went down to the room, perhaps once or twice a month. The custodial and security staff who worked in that general area would have had access to the room, but not the general pool of custodians and security staff who worked elsewhere.
Honestly, it was typical of most places I worked. I am trying to remember a place that did not have similar "limited access" locks on their computer rooms and lab doors. They were an impediment, not high security. (There were buildings and places where highly classified work was done; this was not one of them. I know the difference.)
Carpet - Agreed about carpet in a computer room, but that room was not carpeted.
Vacuum - You got me there. Most of the time the custodial staff used big metal machines with a T-handle and a motor that spun a large flat rotating pad on the bottom. The pads looked abrasive and they somehow cleaned the floor. They look like large household vacuums, so that's what I called them. The people who ran them moved them more from side to side than back and forth. No doubt there is a better, more correct name for the machine. After forty years, I do not remember what my coworker called it, so I called it a "vacuum." I was not there, so I do not know with certainty what machine the custodian actually plugged in to that UPS. Whatever it was, obviously it ran from the UPS, or the custodian would not have kept using that outlet night after night.
That's all I know.
Assuming its linux and you have journalctl setup you can use "journalctl --list-boots".
"Journalctl -b -1" would be the previous boot. -b 0 is the current etc.Press end and go up and see what was going on at the time of the crash.
If this is Windows, you can go through each event in eventviewer find the time where it is starting up and work back and find out what was going on.
Tl;dr you meant something like a floor buffer (all my schools did that daily) and it wasn't a secure server room. You really imagine the rest in your head. If this was just a conversation in person, I'd clarify a few points throughout so I can follow the story. You can also say when you're not sure about something and can just leave it at "custodial appliance" or something. But you seemed confident in a vacuum.
I'm on vacation and shooting the shit with random people daily. There's some errors when people tell stories (there's alcohol involved) and it's often more interesting when someone can correct or confirm different events. It's completely fine to say "oh, yeah, now I remember", it's when people double down on incorrect shit that people will then write you off as full of shit.
Real man don’t use logs or backups, just do a full reset of server - reinstall the OS as probably was compromised 😂
It was a restricted access room. As to "secure", there were multiple layers of security that people had to pass to get to that level, including passing a visual inspection by armed guards of all materials coming and going from the facility.
As usual, you miss the forest by focusing on the color of the fungi on the trees. It changes nothing about the essentials. I was told about what happened during lunches that we often shared at the cafeteria, and it came up more than once.
Stop being a dick. I asked you before, but you don't listen.
Go back and read my post. None of that was being a dick. You really need to let your hard-on for me go soft.
The belief that you have been behaving like a dick is shared by many here. I am not the only one who were subjected to your dickish attacks. It seems to be a regular habit.
Finally, are you incapable of choosing better allusions without obsessing over other peoples' anatomy?
Where did I attack you? Are you being obtuse that you attacked me first? Do you not realize you're being the cunt in this and other situations?
When someone corrects you, they are attacking you?
Dude, you're the one who brought up "dick", not me. You're the one who is fucking obsessed.
This does not deserve a response. I will leave it to others to read Timbo's posts and my own posts and decide for themselves.
Sure. Since jsg doesn't post as much since the Russian aggression, you can be jsg II.
It's probably best if you just stop telling other people's stories, poorly. You're upset someone was politely sympathetic instead of flat out calling you a liar. Ooookay.
That is the same as calling me a liar. @TimboJones was never present for anything I have written about, yet he continually calls my posts into question by attacking the most insignificant details. He does it to others, too.
How many times do people need to say, "You were not there. You insist that you know more than the person who was there. How can you possibly know?"
Timbo's "sanity checks" question the most irrelevant minor details. I worked in that building for two years. I remember it well.
Timbo would question whether I was present for the birth of my own children. I can easily imagine a post from Timbo like, "Birthing rooms use special beds, not chairs, so obviously you could not have been there when your baby was born."
Timbo's petty accusations have become very tiresome. I wish the moderators would step in and put a stop to his frequent bullying of other members, but they don't.
I stand by my statements. Timbo has been a dick, and keeps being a dick, plain and simple.
-> Stop being a dick, Timbo.
You clearly lack people in your life that tell you when you're wrong. You can't handle it. You're throwing the biggest hissy fit over the smallest thing. Stop blaming me for your mistakes.
Your story was essentially, "I once worked at a place that had random reboots at night. Turned out, the custodian was unplugging the server to use the outlet and plugging it back in again".
YOU added the "secure area with limited access", the "vacuum", and a non tripping UPS for an industrial appliance. Also need to ignore that the friggin custodian would unplug something that was running (lights and fans) and plug it back in again and repeatedly ignore that. Including these irrelevant details could be thought to be included to highlight how bad the custodian was. My initial reply was based on what you said! Those "challenges" wasn't arguing with you, it was using the common meme, "...you're gonna have a bad time". E.g, "Your new boss is Elon Musk? You're gonna have a bad time".
The fact nobody else said anything is from lack of interest, not that you tell interesting stories.
Clearly.
This, from someone who knows nothing about the people around me. Someone who feels perfectly free to tell me what I saw, heard, and know (or don't know) when he was not there.
Does he bully his family and the people he works with? Responding to Timbo online is no fun. It must be miserable to live or work around him.
Finally, this is apropos to Timbo:
https://en.wikipedia.org/wiki/Psychological_projection
Yes. That is what I said. You got that right.
Yes, I did. I would have said "described", not "added".
I chose terms like, "secure area with limited access" because that is what it was. To get to that area, you had to pass through a manned security checkpoint, which included an inspection of all containers, even lunch bags. After that, a cipher lock limited access to the computer room. Only certain people knew the combination to the cipher lock.
I already told you that I chose the term "vacuum" to describe the floor cleaning device that the custodian plugged in. I also told you that I do not remember what the team lead called it. A poorly chosen word does not change anything.
The UPS did not complain or trip. If it had, it would not have been a repeating problem and the custodian would not have used it more than once, and we would not be having this discussion. You can anything you want about it, but you were not there. You were never there.
I also chose the term, "custodian". The team lead probably said "janitor". If you pick through the description word-by-word, you can probably find more examples of words that I chose for my description that were not spoken forty years ago.
That is what the custodian did. In my clarification, I commented that the two components were not adjacent to one another. I know where they were, I saw them myself. In case it matters, the room was loud with A/C, machine and fan noise.
You called the details irrelevant and the custodian bad. Not me. I do not know why the custodian felt that they could unplug the server without issues. There could have been a miscommunication about which outlet to use or any number of alternate explanations, but whatever it was, I never learned it. Just because I can't explain why the custodian unplugged it does not mean that it did not happen. You were not there.
Sorry, but I do not understand what you are trying to say here.
Either call me a liar or not. The little nits that you keep picking do not make what I said any less true. I am not a liar, and it is offensive that you continue to treat me that way.
Timbo - Stop telling me what I saw and heard for myself. Stop telling me what happened at a place and time when you were not there.
Timbo - Please stop being a dick.
@emg hope you do realize its just online forum, why going so tense ? I also saw you need moderators to intervene here, why, because @TimboJones is just being little Sherlock Holmes for your story, instead of enjoying it, you made it personal with dick and all.
Take it easy guys. It is not worth getting upset about.
Lol. :facepalm: That fits you in this scenario, perfectly.
And couldn't be bothered to Google it when informed?
https://imgflip.com/memetemplate/112910440/Youre-Gonna-Have-A-Bad-Time
https://www.google.com/amp/s/amp.knowyourmeme.com/memes/youre-gonna-have-a-bad-time
You didn't know about a well known meme, "...you're gonna have a bad time" and incorrectly took me making fun of several bad things YOU wrote about as calling you a liar. That wasn't anything close to my post, it was only you calling yourself a liar. And then you take this as me being a dick. Get a hold of yourself.
I never called you a liar in the first place, you did! It's offensive you keep being obtuse and saying I implied something I never did.
At no time did I ever tell you what you saw or heard. Fucking quote me where I did. You're starting to need psychological help with whatever you think you're seeing and reading, though.
Again, never called you a liar. You called me a dick, many times and then pretended like I said shit that was never said. So, I therefore called you a cunt.
Please stop being a cunt telling ME that I said shit I never said. That's why I told you several posts back to let your hard on for me go soft, it's all in your head.
I am not going to address this one point by point with quotes. No, I will not go to Google to translate your blather, either.
True, you never called me a liar, but you implied it over and over. That is what I find the most offensive, but I have yet to see any restraint or limit to your offensive behavior. I can hardly wait to see your next missive to learn whether you reach new heights.
You bully me and many others on LowEndTalk. That is being a dick.
Timbo - Stop being a dick.
Ok, that's enough. Further escalation will result in serious consequences. Please be civil.
Thread closed.