New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
I use supabase.com managed postgres
Something to check
It depends on many factors such as threads, clients or replication ways ( async or sync ). As I remember, my last benchmark for async was around 3000 tps on OVH SYS with nvme. But for sync over WAN (about 2ms latency) TPS was under 3~500.
Does Supabase tell you IOPS and TPS metrics?
yeah that's a very low TPS volume
Any other experiences anyone would like to share? Let me explain why I'm asking.
At work, our main Postgres database isn't too big-only about 110 GB. Until now, we've been using the local NVME storage on Hetzner Cloud CCX instances. These drives are really fast, with great IOPS, and we've never had performance problems, especially since we were using the more powerful instance types.
The issue was that we used the local-path provisioner storage class to connect the local storage to our Kubernetes pods. Since we run Postgres clusters with CloudNativePG, this setup meant our data wasn't encrypted at rest-and that’s something we needed to fix.
The easiest fix was switching to Hetzner Cloud's block storage, called "Volumes," using their official CSI driver for Kubernetes. We were already using these Volumes for everything except the main database, and they’ve worked well. With Volumes, setting up full disk encryption using LUKS is simple-no extra work needed, just a custom storage class with the encryption enabled. The other options would’ve been more complex, like setting up encryption on local storage using cloud or dedicated servers.
My main concern was whether these Volumes would be fast enough for a production database. They offer up to 5000 IOPS, with bursts up to 7500. That’s a big drop from the 55,000 IOPS we got from local NVME storage. So I was hoping to hear from others about how TPS (transactions per second) relates to storage IOPS in their experiences, just to get a better idea of what to expect.
In the end, I decided to try it anyway-mainly because it’s by far the simplest way to get encryption. Another option could’ve been Longhorn, but it's had some reliability problems in the past, so I'm not fully comfortable relying on it yet.
The good news is, since we use CloudNativePG, switching back to local storage is easy if needed. We can create a new replica cluster and switch over quickly once replication is caught up with minimal downtime.
So far, the results are looking good. Our database load isn’t very high-we peak at around 8000 transactions per second, according to the CloudNativePG Grafana dashboard. Plus, the cloud instances we use have enough memory to keep the whole database in cache, which means fewer reads from disk. Today, we hit 6000 TPS, and during monitoring, I saw we never went above 500 read IOPS or 700 write IOPS. That means we're very unlikely to hit the IOPS limits of the cloud Volumes anytime soon.
Performance feels solid. Honestly, no one has said anything about slowdowns or noticeable delays compared to before. So I'm hopeful we can stick with this setup because it's simple.
If needed, I can always move the WAL to a separate volume or split some parts into different databases to spread out the IOPS load. The only small thing I’ve noticed is that write latency spikes occasionally, but it hasn’t been a real problem so far.
What do you think? Do you see us running into performance issues later on?
This is high-end talk discussion, we only use chicken.
C'mon
Do you really need to encrypt the whole database?
Usually the one that need to be encrypted are PII data. Maybe encryption from the application itself will be simple-enough approach?
Or maybe postgres with TDE like this?
https://doc.pgsty.com/pgsql/kernel/percona/
I'm not sure if CloudnativePG support that
We were doing column level encryption already, but the consultant for the ISO told us that having FDE helps with the audit.
It doesn't, I checked about it when I was looking into the available options.
I use Alloy DB on GCP with an average of 5k TP/s.... and the disks should have 7.500 IOPS/read and 15.000 IOPS/write. DB is around 300GB and query latency is below 1s
But I'll be moving to a PSQL/YugabyteDB/CockroachDB optionwith a disk with stats like below.
Why? Because cloud is too damn expensive.
I prefer running my own ceph cluster.
Have you seen any performance issues with those IOPS?
Nope. Nothing iops rated. At least not with "the disks should have 7.500 IOPS/read and 15.000 IOPS/write."
Nice, thanks!