All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Here Today, Gone When You Exit: Proper Tempfiles in Shell Scripts
raindog308
Administrator, Veteran
Earlier this week I inherited a pile of utterly craptacular shell scripts (running in production, naturally) at work and spend the week essentially rewriting them so they didn't look like they'd been bought on fiverr.
I was told "weird things happen sometimes and we can't figure out a pattern...and the busier we are, the more weird things happen".
It was pretty easy to find the reason why...this identical line was in many scripts that frequently executed:
TEMPFILE=/tmp/tempfile
Every execution was a timing Russian roulette.
Although in one script they used
TEMPFILE=/tmp/tempfile.$$
...and just never rm'd it so there were tens of thousands of tempfiles in /tmp.
As I cleaned up I wrote up a tutorial on how to use tempfiles properly, and how to automatically clean them up when your script exits. It's easy to clean up one, but cleaning up multiple requires some extra thought.
I had a couple solutions in mind (UUID patterns, using an array) but also found a 2009 LJ article with a third approach.
Enjoy: https://lowendbox.com/blog/here-today-gone-when-you-exit-proper-tempfiles-in-shell-scripts/
Comments
BUMP
because that nice tutorial doesn't deserve to drop so quickly
I like a KISS philosophy:
TEMPFILE=/tmp/tempfile.$$
.. along with a daily cron that does..
/usr/bin/find //tmp/tempfile.* -mtime +60 -exec rm {} \; > /dev/null 2>&1
This may suffice.
I enjoyed that article, which reminded me that even with a formal sysadmin training, you can never know all the nuances of bash scripting etc.
(I only remembered enough to do slightly more than basic tasks.)
Nice article
At first, I thought the title was an article about exit scammers. Here today, gone tomorrow.
So handling a single tempfile cleanup with trap is easy.
But to automatically clean up tempfiles, between you and the article we've now listed:
Seems to me like there is a missing feature here :-)
Though now that I think about it, I don't think either perl or python offer a "guaranteed cleanup" when they exit unless you code it yourself.
Not as a built-in.
There are libraries out there that do that sort of thing, but they expect you to call the “clean up” function as you exit, and register a global exception handler (or whatever your language/framework supports to this effect) and/or a signal handler to fire that automatically upon uncontrolled exit. You might still have some left around to remove by other means, due to really uncontrolled exits.
Also take care when using
mktempto pick an arbitrary name to avoid conflicts – for some reason a lot of people assume such files are not created until you output something when they are created as zero-length. I've seen scripts create one initially whether they need it or not, then never remove it assuming that not having used it means it didn't really get created.(or empty, if creating a directory) by
mktempitself. (edit: actually looked at the linked article, and it explicitly shows cleaning up in repose to bash's EXIT signal – note that this is a bashism so don't rely on it unless you explicitly specify bash as the script runner instead of relying on the OS/user's default)Have you tried systemd-tmpfiles? It looks pretty neat.
Yes...I think in other shells you have to trap each signal.
The only negative is that I think you either have to have root or play by root's rules (put files where they'll be cleaned up), or convince someone with root to put in new rules for you. But interesting, and apparently it's been around for quite a while. Here's an overview from 2016:
https://developers.redhat.com/blog/2016/09/20/managing-temporary-files-with-systemd-tmpfiles-on-rhel7
imagine not using pastebin for temporary files
Nice article. If you're having to make complex bash scripts, especially in a team environment, this is a good read too:
https://google.github.io/styleguide/shellguide.html
Today I learned why I don't want to work for Google.
Forcing others to use your preferences for indentation is evil.