Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › How many of you use minio and how do you use it?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

How many of you use minio and how do you use it?

bulbasaurbulbasaur Member
in General

Just what it says on the title -- how many of you use minio, and if so do you use the simple filesystem mode, or the fancier erasure coded/distributed mode?

For those not in the know, it's basically a self hosted version of S3. The reason I ask is that their filesystem mode does some atrocious things that make it unsuitable to use on a storage VPS, but there aren't any good alternatives either outside of using a competing service.

.
  1. How many of you use minio?18 votes
    1. I use minio in filesystem mode
      33.33%
    2. I use minio in distributed mode
        0.00%
    3. idgaf about minio
      66.67%

Comments

  • HalfEatenPieHalfEatenPie Veteran

    So I've used Minio in filesystem mode but it's only for our own development and testing purposes. At the end of the day, it just makes sense for us to use managed solutions like Amazon S3 or Backblaze B2.

    However, in our testing we haven't had to deal with the atrocious things that you mentioned. So a bit more detail regarding that would be appreciated. Although to note, last time we used it in-depth was like 2-3 years ago so maybe I'm a bit outdated.

  • bulbasaurbulbasaur Member
    edited September 2021

    @HalfEatenPie said: However, in our testing we haven't had to deal with the atrocious things that you mentioned. So a bit more detail regarding that would be appreciated.

    The problem is with how filesystem mode is designed:

    • Since S3 clients often use multipart uploads, the completion of the multipart upload literally causes hundreds of parts to be read and joined together, which kills the storage VPS if I try to upload from multiple clients at once.
    • Listing objects is similar -- since S3 APIs are paginated by a maximum of 1000 objects, it has to repeatedly perform listdir() calls, often across a few different directories, only to discard some values because it's over the 1000 objects limit, only for that operation to be repeated on the next call due to the pagination BS, and so on.

    A more efficient approach would have been to use a database and use that to serve paginated object lists. Similarly, it could use the same database to hold the list of multipart uploads and link them logically without merging files, saving the storage VPS somewhat.

    The organization of objects on the local system would not be recoverable without making requests to minio, but it's not something I care about.

  • itsnotvitsnotv Member

    @stevewatson301 said: The reason I ask is that their filesystem mode does some atrocious things that make it unsuitable to use on a storage VPS

    What does it do? I have been using it for a long time for backups on my Kimsufi servers without any issues.

    If you want alternatives, you can install ceph or seaweedfs

Sign In or Register to comment.