All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
MySQL/Pg in the Cloud (as a Service)?
raindog308
Administrator, Veteran
I need a database in the cloud. I have various VPSes and I'd like to coordinate the jobs they run, so I thought I'd have each write to a central DB. I'd also like to store config, capacity planning, etc. data there. Small stuff - probably less than 500M for a long time.
MySQL (or Maria) or PostGreSQL would be ideal but I haven't written any code yet. I do need it to be very available/reliable.
My concerns about self-hosting it on a VPS is high availability/single point of failure/security (MySQL open to the net, though I think MySQL allows you to limit connections from only certain IPs and if not I could do that with iptables). I could cluster but I'm thinking there might be some sort of cloud-based service? I don't mind paying some small amount per month.
AWS RDS (Amazon's hosted MySQL) is $18/month + storage/data at its cheapest (extra small tier), and that's more than I was thinking.
There's Amazon's SimpleDB and the free tier is enough I could probably run for a long time...that's another possibility though it's entirely proprietary: http://aws.amazon.com/simpledb/pricing/ Hopefully there is a perl or python module.
My fallback would be something like MongoHQ or Amazon Dynamo. I see this as more row-based data than document data though.
Comments
Not entirely what you're looking for, but BuyVM does offer offloaded SQL. I don't think its HA though.
The problem I see with hosted MySQL is latency, unless it's on-site such as BuyVM's deployment. Unless your application is not latency sensitive, in which case why not buy multiple SSD VPS's and create your own redundant setup, I don't see too much demand for such a service.
Confused about your requirement, are you looking for a SQL solution, or key/value paris like dynamo or simpledb, or document storage, like mongodb?
The latency between the DC could be noticeable to the end users though, unless you host everything in the same AZ.
Google has "Cloud SQL". I played with AWS RDS and Google Cloud and it looks like the cheapest I can get this is something like ~$20/month (micro instance, 1GB of storage, 15GB transfer) from either, or ~$15/month from AWS with a one-year reserved instance. Hmmm...
SimpleDB is free for 1GB and 25 hours...I'd have to dig into how those hours are calculated but it's just actual query running time. Not sure if that's cumulative time or "any hour in which a query runs costs you one hour". Still have to pay transfer.
latency and security and your big caveats here
Well clearly there's a ton of demand, because Amazon, Google, etc. offer it :-)
Not looking to offer such a service, just use it. Multiple redundant SSD VPSes would be expensive - I was hoping for something under $10/month.
This use is not latency-sensitive...it's batch jobs.
Ideally SQL. I could probably make NoSQL work but it'd be a lot easier with SQL (for aggregate data, etc.)
How large is your Database? 2 VPSes would probably suffice, if more uptime is required your budget is not high enough for a truly mission critical application.
RamNode 128's are ~$15/yr. I'm running a fairly large database on one of mine.
Very small - under 1GB. It's all job state info, and then capacity planning/trending data.
Yeah, rolling my own with MySQL Cluster is one way to go. Hmmm...going to research more.
Other stuff I found:
All of that is a lot more than I want to pay. I was thinking of something like MongoHQ's free tier, only for MySQL.
Grab 2 RamNodes then, one in SEA one in ATL and only run MySQL on those VPSes and you should have no problem. You could look at another cheap SSD provider, I can't recommend any others off the top of my head but I'm sure they're out there.
https://www.digitalocean.com/community/articles/how-to-set-up-master-slave-replication-in-mysql
I'm using my 128 RamNode to run my main Observium, it's database is around 2GB and it also runs some other services aside from SQL.
Found one more - ClearDB ($9.99/month)
But yeah maybe I'll just roll myself...though I'll go cross-provider because I'm paranoid :-)
@nunim, the tut you linked only solves part of the problem, actually.
The other parts are:
I was hoping not to have to dig into the MySQL docs...sigh...
I haven't thoroughly read the docs, but are you running ndbcluster on a 128MB VPS? Because from what I've read it's a memory pig and I don't see how you can have a fault-tolerant MySQL setup without it.
Do you need mysql-proxy? Because then that's another SPOF you have to fix...
Could always try taking 2 or 3 LES boxes if none standard ports are OK with you
cost is the same as a beer and you have some redundancy, happy to work with you on putting auto fail over/ redirect rules on the HAproxies if needed.
http://www.hostdepartment.com/databases/mysql-hosting/
Try BuyVM off loaded then buy some NJ/NY vps' near by for low latency
Not trying to derail your thread, but I have a similar question and don't want to waste a thread over it. Is there such a thing as "mirroring MySql"? I've looked into and it "EXISTS" but it sure isn't easy to make. I've also heard answer along the lines of "it's only available for enterprise solutions." MySql seems to be my bottleneck and if I could have MySql mirror itself life would be bliss.
I've looked into things like MariaDB and that MariaDB supports a "cluster" sort of model, but I wanted to see if anyone else has heard of anything or knows anything. I've also looked at NoSql solutions but it's just a big no no as it's a pain to implement (in my opinion).
@skybucks100 "mirroring MySql" = replication
and it has existed in the GA for YEARS.
It looks like OP is running some ETL and analysis on a very small dataset? I would go for AWS RDS or Google's CloudSQL solution, as the upfront endeavor is really not worthy.
If you'd like roll-you-own solution, try master/slave set up in the MySQL, then find the your conform zone of HA and latency.
I've tried replication (I call it mirroring), but it seems everytime I try it I manage to break something. It's also a pain to get setup and configured. I'll give it a go again, but I was wondering if there was a solution that was more "plug and play" and a bit easier to use/configure.
Edit: Ignore the first suggestion. I only noticed the OP's requirement of 'row-based data' just now.
For the OP's use case, wouldn't a hosted K/V store like a Redis Saas subscription do as well?
I see some with a free tier of 5-25MB or so. Paid tiers start at about $7/m.
Google around.
As for Mysql multi-master replication(aka 'mirroring' solution), Check out Percona's free 'XtraDB cluster' product (It's just mysql + Galera's synchronous M-M replication patches)
It's a drop-in replacement for your existing mysql installation.
Write to ANY node , Read from ANY node, since all COMMITs are synchronized.
Prometeus also has a very cheap mysql offload with ssd and stuff, however, it is in Milano and limited to 100 connections, but the latter issue does not seem to be a problem.