New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
@ECNetworks
"With x GB RAM you don't need swap" is nonsensical because for some cases even 256 MB RAM may be a generously sufficient amount while for others 16 GB are damn tight.
That said, I personally generally disable swap and I can tell based on experience that's not at all a problem if your RAM is large enough.
Reason: a) I've seen enough cases where shitty apps pervertedly wasted memory because "there's so much of it" (seemingly) or because they just love to play with vmem or think they are smarter than the kernel.
And b) because it's multiple orders of magnitude slower.
The usual explanation is something along the line of "rarely used memory pages" etc. Well, problem is that that's not the whole story. No matter who and how rarely used, something that uses vmem sloooooows down the hole box incl. the services that are really important and sensitive. (Keep in mind that we're talking servers here, not desktops).
So, a better way is to streamline, trim, and tune your installation. You don't use NFS, samba, cups, etc? Well, then get rid of them.
That said, I strongly suggest that less experienced admins just eat the soup the way it's served and don't touch things without really understanding the consequences (and being able to track down problems themselves).
For noobs I personally see swap as something of a last resort safe net.
Reason: a) I've seen enough cases where shitty apps pervertedly wasted memory because "there's so much of it" (seemingly) or because they just love to play with vmem or think they are smarter than the kernel.
And b) because it's multiple orders of magnitude slower.
I agree that RAM is so cheap these days that using swap is not really worth it, but set up correctly and with the just right kind of swapiness (0 is usually a safe bet) there should be nothing to worry about and some resources can be freed with no other cost than "dump to disk and forget it" for the really badly written or too cautious apps. If the ram will be needed and there is no contention for it, it will likely make it back to the real ram and stay there.
Trusting automated processes and leaving the kernel to care is not usually something I am fond of, but that goes for my usage, i cannot predict what the customers will do and how well will they setup their boxes, I must assume the worst and that the kernel will do a better job than them. Also, on servers which run for days at home (until a longer power outage that the UPS wont do for) setting up swap is a safe bet because the heavy disk will only go for a few seconds at bootup, there is almost no disk activity on the main disk afterwards, so much so that I use usb disks and even sticks for root and swap without issues there. Also, they are far from having tons of ram, I mean I have those servers too, but they are heavy on the power usage side.
Are we talking about this?
I apologize if I was not stating clear, as I was on my phone, I was sleepy, and I'm not a native speaker.
Please allow me to explain a little more.
For those who know programming, you know that when you need a memory space, you will need to allocate one before using. For the C family, you use
malloc/newto claim the memory, and then read or write the memory space.The memory you claimed is
committed memoryand the memory you wrote isused memory. If you have a large amount of committed memory, you won't be able to see that on yourfreestatus or you task manager (if on Windows), but this will still prevent you from allocating more memory and will cause crashes or OOM.This is a random picture I found on the Internet. You can observe that the committed is 1GB and 900MB is used. That being said, if this happens on a computer with only 1GB memory and no swap, you are out of memory and your apps start to crash AND you still have 100MB memory to spare.
I don't comment on your understanding how memory allocation, management, and usage work.
And I don't know a lot more than my grandma about Windows because of its utter irrelevance in the server world.
But at least on unixoid systems, No, your applications don't simply "crash" in out of memory situations.
From a programmers view the game goes like this: You have a certain amount of "fixed" memory needs and then some dynamic needs/allocations, usually related to task that need be done. Things like serving clients, reading and working on files, etc.
Now if, say a web server suddenly has a very high number of requests/connections it obviously needs more memory because each connections handler will need some memory. If the call to allocate that memory fails, all that happens in a well behaving server is that it gets slower by limiting itself to a rate of served connections it can handle (incl memory requirements).
Moreover the very problem you describe can not be solved by swap. Because swap isn't limitless; it's as limited as RAM. All you'll do is to stretch the problem somewhat (and to get the whole system even slower).
Now it is clear. I was thinking you are having a brain fart kind of thing because you seem to know what you are talking about, in general, but that theory i explained does exist.
While C people are caring for ram usage most of the time, at least the old timers which know why a name like microsoft was a good idea at the time, java people do not really care and the implementation is also wasteful. It does not matter that much today, but in the times of 64-128-256 MB ram laptops, running java was a daring enterprise, the computer used to "'purr" for minutes before something appeared to be working.
I see now. You're not insane and probably know a good it more than I do about memory allocation with C. Overcommit so often refers to something more simple here it's easy to mistake
What I was talking about, was actually one kind of memory-leaking-behavior that happens almost everywhere. For example, innodb engine inside MySQL, Java virtual machine, etc.
These system tend to allocate a large portion of memory and not use them at that moment, but later. MySQL, with innodb engine, will try to allocate 600MB upon start, but will only use 50MB of that. The consequence is, you won't be able to start it on a 256MB VPS without swap. With swap, however, it will start and only consume 50MB out of 256MB, which looks good.
The overcommitment policy does not only applies to Windows, but Linux kernel as well, unfortunately. That's why in
topyou seeVIRT (committed memory)andRSS (memory in use). You were talking about the latter, but I was mentioning the former.This is my understanding about the system. Please correct me if I'm wrong. Thanks.
A quick example was here.
On a 256 MiB machine without swap:
Allocated 2933MiB, Used 98MiB before killed
With 539 MiB swap:
Allocated 2933MiB, Used 635MiB before killed
To prevent the program being killed when only using 98MiB out of 256MiB, a swap is suggested.
Java and MySql? Confirms my understanding that both are lousily designed (and implemented).
I think anyway that this whole "desired" or "committed" memory idea is nonsensical. The correct way would be to a) adapt OS buffers to the current memory situation (rather than playing commitment games) and to b) design programs properly, i.e. to deal with low memory situations.
And, well noted, memory scarcity is anyway largely due to idiotic design and wasteful attitude.
Well, whatever, I personally didn't experience any of the problems discussed here, probably in part because I virtually never use windows and rarely use linux, avoid java like the pestilence and don't use MySql (nor PHP or other "cool" toys).
But I agree that keeping swap on probably is sane advice for, uhm, less experienced admins and people who believe in programs having gpl license being a replacement for sound engineering.
Have a nice weekend everyone