New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Morning random thoughts - Amazon S3/EC2 script
Had this idea during my morning shower but I do not have the scripting knowledge to do it, so if anyone wants to take the task and help the community that would be great.
Basically the idea is, since we can use S3 and EC2 separately, can we use a script that sits on my own LEB and scheduled by cron so that it will create a EC2 instance on the fly (using the API), send the files I would like to backup from my LEB to S3, then shut down/delete the EC2 instance so that I do not have to pay for it?
That way we need a EC2 instance for 5 minutes every time I am backing up my VPS + the amount of space I have to pay for a S3, would that be nice?
Comments
This seems like a bad idea. Why is EC2 needed at all?
s3cmd http://s3tools.org/s3cmd
I thought S3 is just like a hard drive, EC2 is the actual "computer"?
S3 is plain old storage, independent of EC2.
You don't need EC2 in this case
You don't need EC2 in this case
How do you connect to S3?
this seems really promising, looking into this.
I've been using this for a long time for various things (backups for customers and preparing content for CloudFront). It works really well and now that they have fixed some bugs relating to uploading large files it's definitely a must have if you need to do anything with S3 on Linux.
I've been using s3cmd to backup all my websites to S3, will be useful to recover websites should I need to.
they should make glaciercmd too.
Why not use dejadup to back it up, or mount S3 with s3fs as a local disk?
https://github.com/LordGaav/glaciercmd
Nice william
I am actually using that to keep 100GB of backups sync using rsync, s3fs and crontab
Thanks, should look into that today
In my testing s3cmd is far superior to s3fs but maybe its just me. Also s3cmd has a sync feature which almost replicates rsync and your not wasting resources, s3 requests, and troubleshooting s3fs.
s3cmd can upload and flag as reduced redundancy as well which can cut your costs as well.