Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Using gdrive as storage for VPS
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Using gdrive as storage for VPS

AstroAstro Member

Is anyone using gdrive mounted locally as extended storage for nextcloud, plex etc? Theoretically seems possible with rclone but wondering if there are any issues with since I imagine it'll create a lot of pull requests to google servers.

Comments

  • drizbodrizbo Member

    I used to have it mounted for plex, and was working pretty fine. I was even seeding torrents directly off it. There were issues with dropping connection sometimes when seeding, but for plex it worked pretty ok.

  • AstroAstro Member

    @drizbo said: There were issues with dropping connection sometimes when seeding, but for plex it worked pretty ok.

    Did you ever find a workaround for it?

  • adwsislifeadwsislife Member
    edited July 2022

    @Astro said:

    @drizbo said: There were issues with dropping connection sometimes when seeding, but for plex it worked pretty ok.

    Did you ever find a workaround for it?

    It wont work good because in seeding/torrent, files are distributed as pieces (1MB/2MB/4MB pieces).
    When you mount gdrive, files are just available via network, but not actually present in server disk. so........ when a random piece of a file is demanded, rclone has to fetch it from Gdrive. Hence the dropping connections.

    Regarding plex, it will work great. Because in plex. it needs full file to play a movie/series rather than random pieces. So....you can use rclone to cache files which plex can use to play.

    You can use the same caching technique for seeding as well but it will take lot of time to do a hash check. Not recommended

    Hope I was able to explain it properly :sweat_smile:

    Thanked by 1Logano
  • darkimmortaldarkimmortal Member
    edited July 2022

    Have done this with slower backends than google drive in the past. For streaming it is passable. For seeding you will want to dedicate a torrent client to rclone and/or avoid the single threaded clients like Transmission. The tuning and caching options in rclone are your friend - took a large (100s of GB) cache for me

  • offtopic, what is the best caching method for SFTP mounting?

  • dosaidosai Member

    Nope, never had any issues. Been using for years now.

  • AstroAstro Member

    @adwsislife said:

    @Astro said:

    @drizbo said: There were issues with dropping connection sometimes when seeding, but for plex it worked pretty ok.

    Did you ever find a workaround for it?

    It wont work good because in seeding/torrent, files are distributed as pieces (1MB/2MB/4MB pieces).
    When you mount gdrive, files are just available via network, but not actually present in server disk. so........ when a random piece of a file is demanded, rclone has to fetch it from Gdrive. Hence the dropping connections.

    Regarding plex, it will work great. Because in plex. it needs full file to play a movie/series rather than random pieces. So....you can use rclone to cache files which plex can use to play.

    You can use the same caching technique for seeding as well but it will take lot of time to do a hash check. Not recommended

    Hope I was able to explain it properly :sweat_smile:

    That makes sense thanks! So I guess the best way to do it is download locally and transfer after seeding.

    It should be okay for nextcloud right?

  • AstroAstro Member
    edited July 2022

    @darkimmortal said:
    Have done this with slower backends than google drive in the past. For streaming it is passable. For seeding you will want to dedicate a torrent client to rclone and/or avoid the single threaded clients like Transmission. The tuning and caching options in rclone are your friend - took a large (100s of GB) cache for me

    Is there a place where I can find a way to get the tuning and caching in order?

    @dosai said: Nope, never had any issues. Been using for years now.

    >

    What is your setup like?

  • drizbodrizbo Member

    I had some scripts that remounted my shit when it happened and I skipped hash checks in rtorrent.

    I was just trying to seed like 100TB off the cloud but it's not something i'd recommend as its not stable, but it is working. But just seeding, I wasnt downloading directly on it.

    So if you're using it for plex, just download movies to local storage and then transfer it to cloud. I think all torrent clients have move/copy upon finish function, and most have possiblity to run own scripts on any event.

  • Why do you prefer torrents and not Usenet, with up to 1gbit speeds and without all the seeding crap involved? The yearly plan is like $30.

  • dosaidosai Member

    @Astro said:

    @darkimmortal said:
    Have done this with slower backends than google drive in the past. For streaming it is passable. For seeding you will want to dedicate a torrent client to rclone and/or avoid the single threaded clients like Transmission. The tuning and caching options in rclone are your friend - took a large (100s of GB) cache for me

    Is there a place where I can find a way to get the tuning and caching in order?

    @dosai said: Nope, never had any issues. Been using for years now.

    >

    What is your setup like?

    Plex and gdrive + rclone mount as media source.

  • drizbodrizbo Member

    @luckypenguin said:
    Why do you prefer torrents and not Usenet, with up to 1gbit speeds and without all the seeding crap involved? The yearly plan is like $30.

    Because 1000x more content on trackers, even higher speeds, especially on actual content. A lot more releases, higher quality encodes. I feel nice sending out petabyte of data too.

    You need to be on proper trackers though, these public trackers dont even remotely compare to the better private ones.

  • @Astro said:
    Is anyone using gdrive mounted locally as extended storage for nextcloud, plex etc? Theoretically seems possible with rclone but wondering if there are any issues with since I imagine it'll create a lot of pull requests to google servers.

    rclone yes, hella slow tho. and also if i remember there was a cap for the pull/push.

  • what about onedrive?

  • dosaidosai Member

    @Deepak_leb said:
    what about onedrive?

    I've tried onedrive with rclone mount but it is slow for me.

  • It works with Plex when configured correctly along with rclone and rclone cache.

  • stefemanstefeman Member
    edited July 2022

    rclone + mergerfs makes it work like normal filesystem. Caching included.

  • You can mount gdrive to your VPS as a disk. with countless softwares

  • @Deepak_leb said:
    what about onedrive?

    Holy fuck it's slow to the hell (testbench: Azure Japan East D8as_v5) (I WAS EVEN USING AZURE)

  • will google flag you as a pedophile if you store midget porn in gdrive?

  • Google drive and rclone works for alot of stuff, including nextcloud. But you want to install nextcloud on your vps, and use a addon for external storage. It will time out othervise because it makes 1000s of small files all the time when uploading to it

  • Rclone works fine I even used to use a vps to make my google drive into a website and downloads work fine as well

  • @selfhoster said:
    Google drive and rclone works for alot of stuff, including nextcloud. But you want to install nextcloud on your vps, and use a addon for external storage. It will time out othervise because it makes 1000s of small files all the time when uploading to it

    Nextcloud didn't work for me they rate gdrive at 2 uploads per second I think

  • @Harmony said:

    @selfhoster said:
    Google drive and rclone works for alot of stuff, including nextcloud. But you want to install nextcloud on your vps, and use a addon for external storage. It will time out othervise because it makes 1000s of small files all the time when uploading to it

    Nextcloud didn't work for me they rate gdrive at 2 uploads per second I think

    Nah, that depends how you set it up.
    Also, do you really need more than 2 uploads / secound for nextcloud?

  • Have you considered Hetzner's Storage Box mounted via CIFS? It would be better performing and more reliable IMO.

Sign In or Register to comment.